2023-06-22T17:36:58.506 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-22T17:36:58.507 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848 --name yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/orig.config.yaml 2023-06-22T17:36:58.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T17:36:58.647 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848 branch: wip-yuri4-testing-2023-06-21-0846-quincy description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} email: yweinste@redhat.com first_in_suite: false job_id: '7311848' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri4-testing-2023-06-21-0846-quincy ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-06-21-0846-quincy sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9037 sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2023-06-21-0846-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c targets: smithi039.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe9OFf/6dh5inwTSUjkG0HNiVExbLvI5xRjOsAWZcv8Kb/7u0naxIwsYjWC7YNUWCv44q3yyhAV/h/QxeBsEOaFH1azLrTZbMBDz6HIrr/vw4iDZQmVdTWSeyxmDpfcogvYfV5oMPVAnW6K/35J8gbEhI2mBAijc9rt5z4MKdc5jMmaMFhOmsTGkFqNMDg1jEwMjoFpHvfkWYEYhySYdmssODxy6u1vRwWUV3nizSTmowz+N2AhbLXwUkgkyEo1bMOHWimjcJqUGOisfoHIITTCjXXaOsxdxxsmQqHZd8kwcZC3p3RSzppg1htlEXnlkfT8JYvq1kTa4NMzGShv/zR+trwJIqiwNg5zQFS7TKRgpSPTvcdkzY1X7KmSAKJT2rLRDPzgy2qG0Sp1CKtTfoBEHRfjsghC4vuWkaUESIK66llnrzdYo/cFUNiqG0QsB+Spj2ZbGz9tyf/4/ig9gAI2C1Z5eyZnBbDc+Irx+xWn6giTO60IqJlL+/T9XMgO0M= smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPijkdxgmpxaM2Hf+dEfbZJu1m48fjhao5gRK61VKJ4nYxXgoThOA3lR4BrmwCHdSiut3679K8GeK5t6yFBTU19ukf+irDuKwsyI7tmi3QrfhO83sedhLWgADiu9Jv+cDVUOFenhejI9ddgI4tzUXDo9g3RrQjEHqYBv1e8YBWcc3XZvnawpNxh1sn0VQX9+Y1z5baP0Yo8GT0pM5vNehWib1kMJVtVSWol/I+ZzYTVZR3SMZHYpAV4/uX+11mKKyBz9b8bj+ICDlaQI0ftLFGevk6ybr87UMG4SJEEpG1uYNMYJjEC4CZAHBFa5b9LlcjH7KVjpjS8mnYFTmA1PFoLAQVVwBx/A9D8MUydrZRfmYWp011IfXgryoQjTJWaorIhnGyE+5Ul7LZYxngmGmOnuMiS4YWzy7YeMEdc15Of2hFfMYnP0VAojJirAq2ILkKG23LCXSp2ypbceVuiImnDHbbXUSo/iiikoThKOTSXE6/il9xpKk/3BM26e69yU= smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPW6QPBFRJ8SvhStxm6y5q211RlqOxyZcLIdkrKt2gUy9c3DZC0sn98VqvxjuSNV0U1mrc0olLrMsvfgfsMW0i0G2ELBGZURx+ynEO+MnCrp3kLat4P9qkT9rUkAqrYL6sy/jsLWh9DDX701oAZ6qqxAnOBpygcxthciZX//A9nalaoI/CZ1yvfsRaIawhN0HN9PanKHM3AoafR7v5YCMGLYDv16p1wB9spmY10sPMQB4NNf8R9DxlYo84DffMz/JbGzd5/HV6fg0HGJnPKvuOrfROWdvpCVmDXGbYdYL7CgUBs+KWss0qJj018ZUwL3r8aruv6E/oMxuYQnIqmipfaEm/RkRUH0CUFwMGMjkK814PfA+FBMYpn/weFm6ELBDywYeA8zJXiueBFCGSDEn6Q6EDHLf0d3LVIQ1Pmsi57jL7h7jyiQs6Rv/FlFkW68IJsr5UgWB4nrdiA953RMUoNJBSGaqoqg0kG8vgcUvw38t1VIFATIuP5jYRLNfOMcs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-22_15:25:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-22T17:36:58.647 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa; will attempt to use it 2023-06-22T17:36:58.648 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks 2023-06-22T17:36:58.648 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-22T17:36:58.650 INFO:teuthology.task.internal:Checking packages... 2023-06-22T17:36:58.676 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '42ddc46e6e6582d8e29a5a982a81886fe028be2c' 2023-06-22T17:36:58.676 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-22T17:36:58.677 INFO:teuthology.packaging:ref: None 2023-06-22T17:36:58.677 INFO:teuthology.packaging:tag: None 2023-06-22T17:36:58.677 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-22T17:36:58.677 INFO:teuthology.packaging:sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:36:58.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-22T17:36:58.915 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-719.g42ddc46e 2023-06-22T17:36:58.916 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-22T17:36:58.923 INFO:teuthology.task.internal:no buildpackages task found 2023-06-22T17:36:58.923 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-22T17:36:58.969 INFO:teuthology.task.internal:Saving configuration 2023-06-22T17:36:58.987 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-22T17:36:59.002 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T17:36:59.030 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:30:57.946729', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHQSDLFfWINN90Hkj4dIrCqxMu4JofM3A7ecexZnInTuJc7OAmo2PR/zza9HllRUw2Wk5S0aWDDCIbX5sy9Fs8M='} 2023-06-22T17:36:59.055 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:30:57.944641', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBi/jdeVB9N8GDUFBjRpmTXh0Ed558ROuzepPO870Dx34pYcvn0xw5npl25HGBRLklnGnQWl2wOkYv8OsiXgmkA='} 2023-06-22T17:36:59.075 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:30:57.948371', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDYPaglYdUg8eJv5cR3y2NvxsGlbuy/BU/g+0Pnf8Z/s5RoMgSzvItTpKoohMzpePAztPJSUuN592YYcl8Hrmuc='} 2023-06-22T17:36:59.076 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-22T17:36:59.082 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-06-22T17:36:59.083 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-06-22T17:36:59.083 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-06-22T17:36:59.083 INFO:teuthology.run_tasks:Running task console_log... 2023-06-22T17:36:59.223 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f012f3bc0d0>, signals=[15]) 2023-06-22T17:36:59.225 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-22T17:36:59.231 INFO:teuthology.task.internal:Opening connections... 2023-06-22T17:36:59.232 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2023-06-22T17:36:59.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:36:59.302 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi101.front.sepia.ceph.com 2023-06-22T17:36:59.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:36:59.371 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2023-06-22T17:36:59.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:36:59.442 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-22T17:36:59.449 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2023-06-22T17:36:59.467 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2023-06-22T17:36:59.468 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-06-22T17:36:59.525 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:36:59.525 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:36:59.525 INFO:teuthology.orchestra.run.smithi039.stdout:ID="rhel" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="fedora" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8.4" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:36:59.526 INFO:teuthology.orchestra.run.smithi039.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:36:59.527 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:36:59.527 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:36:59.527 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:36:59.527 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:36:59.527 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:36:59.527 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:36:59.528 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-06-22T17:36:59.552 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-06-22T17:36:59.569 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-06-22T17:36:59.570 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-06-22T17:36:59.625 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:36:59.626 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:36:59.626 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel" 2023-06-22T17:36:59.626 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora" 2023-06-22T17:36:59.626 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.4" 2023-06-22T17:36:59.626 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:36:59.626 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:36:59.627 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:36:59.628 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:36:59.628 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:36:59.628 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-06-22T17:36:59.652 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-06-22T17:36:59.669 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-06-22T17:36:59.670 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-06-22T17:36:59.725 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:36:59.726 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:36:59.726 INFO:teuthology.orchestra.run.smithi133.stdout:ID="rhel" 2023-06-22T17:36:59.726 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="fedora" 2023-06-22T17:36:59.726 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8.4" 2023-06-22T17:36:59.726 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:36:59.727 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:36:59.727 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-06-22T17:36:59.727 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:36:59.727 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:36:59.727 INFO:teuthology.orchestra.run.smithi133.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:36:59.727 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:36:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:36:59.728 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:36:59.728 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:36:59.728 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:36:59.728 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:36:59.728 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-06-22T17:36:59.752 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-22T17:36:59.764 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-22T17:36:59.838 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-22T17:36:59.839 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2023-06-22T17:36:59.841 DEBUG:teuthology.orchestra.run.smithi101:> test '!' -e /home/ubuntu/cephtest 2023-06-22T17:36:59.843 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2023-06-22T17:36:59.859 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-22T17:36:59.869 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-22T17:36:59.869 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2023-06-22T17:36:59.898 DEBUG:teuthology.orchestra.run.smithi101:> test -z $(ls -A /var/lib/ceph) 2023-06-22T17:36:59.901 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2023-06-22T17:36:59.913 INFO:teuthology.orchestra.run.smithi039.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-22T17:36:59.915 INFO:teuthology.orchestra.run.smithi101.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-22T17:36:59.917 INFO:teuthology.orchestra.run.smithi133.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-22T17:36:59.918 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-22T17:37:00.001 INFO:teuthology.run_tasks:Running task kernel... 2023-06-22T17:37:00.017 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-06-22T17:37:00.018 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-06-22T17:37:00.018 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-06-22T17:37:00.019 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-22T17:37:00.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:37:00.036 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-06-22T17:37:00.283 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1c622d92b6f18ce44a7c33802686ff61c3c273ab 2023-06-22T17:37:00.283 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1c622d92b6f18ce44a7c33802686ff61c3c273ab"... 2023-06-22T17:37:00.283 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2023-06-22T17:37:00.300 INFO:teuthology.orchestra.run.smithi039.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-22T17:37:00.301 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs 1c622d92b6f18ce44a7c33802686ff61c3c273ab 2023-06-22T17:37:00.301 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-06-22T17:37:00.301 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum upgrade -y linux-firmware 2023-06-22T17:37:01.738 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-06-22T17:37:01.739 INFO:teuthology.orchestra.run.smithi039.stdout:Unable to read consumer identity 2023-06-22T17:37:05.457 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-22T17:37:07.158 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 16 MB 00:01 2023-06-22T17:37:10.619 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-22T17:37:11.547 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-06-22T17:37:11.548 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2023-06-22T17:37:11.549 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-06-22T17:37:11.673 INFO:teuthology.task.kernel:Downloading kernel 1c622d92b6f18ce44a7c33802686ff61c3c273ab on client.0... 2023-06-22T17:37:11.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c622d92b6f18ce44a7c33802686ff61c3c273ab 2023-06-22T17:37:11.883 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/1c622d92b6f18ce44a7c33802686ff61c3c273ab/centos/8/flavors/default/x86_64/ 2023-06-22T17:37:11.884 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_rc5_g1c622d92b6f1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/1c622d92b6f18ce44a7c33802686ff61c3c273ab/centos/8/flavors/default/x86_64/ --input-file=- 2023-06-22T17:37:11.886 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi039.front.sepia.ceph.com to complete... 2023-06-22T17:37:24.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22 17:37:24 URL:https://2.chacra.ceph.com/r/kernel/testing/1c622d92b6f18ce44a7c33802686ff61c3c273ab/centos/8/flavors/default/x86_64/kernel-6.4.0_rc5_g1c622d92b6f1-1.x86_64.rpm [379883408/379883408] -> "/tmp/kernel.x86_64.rpm" [1] 2023-06-22T17:37:24.982 INFO:teuthology.orchestra.run.smithi039.stderr:FINISHED --2023-06-22 17:37:24-- 2023-06-22T17:37:24.982 INFO:teuthology.orchestra.run.smithi039.stderr:Total wall clock time: 13s 2023-06-22T17:37:24.983 INFO:teuthology.orchestra.run.smithi039.stderr:Downloaded: 1 files, 362M in 13s (28.4 MB/s) 2023-06-22T17:37:24.984 INFO:teuthology.task.kernel:Installing kernel 1c622d92b6f18ce44a7c33802686ff61c3c273ab on client.0... 2023-06-22T17:37:24.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-06-22T17:37:26.807 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying... ######################################## 2023-06-22T17:37:27.188 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing... ######################################## 2023-06-22T17:37:27.333 INFO:teuthology.orchestra.run.smithi039.stdout:Updating / installing... 2023-06-22T17:38:07.810 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-6.4.0_rc5_g1c622d92b6f1-1 ######################################## 2023-06-22T17:38:29.836 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-06-22T17:38:29.836 DEBUG:teuthology.orchestra.run.smithi039:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-06-22T17:38:29.883 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/System.map-6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:29.883 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/config-6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:29.883 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:29.883 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-06-22T17:38:29.884 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/crc32c-intel.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-06-22T17:38:29.885 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/serpent-avx2.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-06-22T17:38:29.886 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel/cpu 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel/cpu/mce 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm 2023-06-22T17:38:29.887 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm/kvm-amd.ko 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm/kvm-intel.ko 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/arch/x86/kvm/kvm.ko 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/ansi_cprng.ko 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/anubis.ko 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/arc4.ko 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx 2023-06-22T17:38:29.888 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_memcpy.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_pq.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_raid6_recov.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_tx.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/async_xor.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/async_tx/raid6test.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/authenc.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/authencesn.ko 2023-06-22T17:38:29.889 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/blake2b_generic.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/blowfish_common.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/blowfish_generic.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/camellia_generic.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cast5_generic.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cast6_generic.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cast_common.ko 2023-06-22T17:38:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/ccm.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cmac.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/crc32_generic.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/crypto_engine.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/crypto_user.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/cts.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/des_generic.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/dh_generic.ko 2023-06-22T17:38:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/echainiv.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/essiv.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/fcrypt.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/khazad.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/lz4.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/lz4hc.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/md4.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/michael_mic.ko 2023-06-22T17:38:29.892 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/pcbc.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/pcrypt.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/rmd160.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/seed.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/serpent_generic.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/tcrypt.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/tea.ko 2023-06-22T17:38:29.893 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/twofish_common.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/twofish_generic.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/vmac.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/wp512.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/xcbc.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/xor.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/crypto/xxhash_generic.ko 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers 2023-06-22T17:38:29.894 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/acpi_ipmi.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/acpi_pad.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/custom_method.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/ec_sys.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/platform_profile.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/sbs.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/sbshc.ko 2023-06-22T17:38:29.895 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/acpi/video.ko 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/acard-ahci.ko 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/ahci_platform.ko 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/ata_generic.ko 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/libahci_platform.ko 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_acpi.ko 2023-06-22T17:38:29.896 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ali.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_amd.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_artop.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_atiixp.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_atp867x.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_cmd640.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_cmd64x.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_cypress.ko 2023-06-22T17:38:29.897 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_efar.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt366.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt37x.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt3x2n.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_hpt3x3.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_it8213.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_it821x.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_jmicron.ko 2023-06-22T17:38:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_marvell.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_mpiix.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_netcell.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ninja32.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ns87410.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_ns87415.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_oldpiix.ko 2023-06-22T17:38:29.899 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_opti.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_optidma.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_pcmcia.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_pdc2027x.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_piccolo.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_rdc.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sch.ko 2023-06-22T17:38:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_serverworks.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sil680.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sis.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_sl82c105.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_triflex.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pata_via.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/pdc_adma.ko 2023-06-22T17:38:29.901 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_inic162x.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_mv.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_nv.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_promise.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_qstor.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sil.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sil24.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sis.ko 2023-06-22T17:38:29.902 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_svw.ko 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_sx4.ko 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_uli.ko 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_via.ko 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ata/sata_vsc.ko 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/auxdisplay 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/auxdisplay/ks0108.ko 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/base 2023-06-22T17:38:29.903 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/base/regmap 2023-06-22T17:38:29.904 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/base/regmap/regmap-i2c.ko 2023-06-22T17:38:29.904 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/bcma 2023-06-22T17:38:29.904 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/bcma/bcma.ko 2023-06-22T17:38:29.904 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/aoe 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/aoe/aoe.ko 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/brd.ko 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/drbd 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/drbd/drbd.ko 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/floppy.ko 2023-06-22T17:38:29.905 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/loop.ko 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/mtip32xx 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/nbd.ko 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/null_blk 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/null_blk/null_blk.ko 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/pktcdvd.ko 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/rbd.ko 2023-06-22T17:38:29.906 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/virtio_blk.ko 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/xen-blkback 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/xen-blkfront.ko 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/zram 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/block/zram/zram.ko 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hangcheck-timer.ko 2023-06-22T17:38:29.907 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/amd-rng.ko 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/intel-rng.ko 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/via-rng.ko 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/hw_random/virtio-rng.ko 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi 2023-06-22T17:38:29.908 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_si.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/lp.ko 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/mwave 2023-06-22T17:38:29.909 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/mwave/mwave.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/ppdev.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tlclk.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_atmel.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_infineon.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_nsc.ko 2023-06-22T17:38:29.910 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_tis.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/uv_mmtimer.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/char/virtio_console.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/p4-clockmod.ko 2023-06-22T17:38:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/powernow-k8.ko 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/cpufreq/speedstep-lib.ko 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/ccp 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/ccp/ccp.ko 2023-06-22T17:38:29.912 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_common 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/padlock-aes.ko 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/padlock-sha.ko 2023-06-22T17:38:29.913 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/virtio 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dca 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dca/dca.ko 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/devfreq 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma 2023-06-22T17:38:29.914 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/dw 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/dw/dw_dmac.ko 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/ioat 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/dma/ioat/ioatdma.ko 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/amd64_edac.ko 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/e752x_edac.ko 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/edac_mce_amd.ko 2023-06-22T17:38:29.915 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i3000_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i3200_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i5100_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i5400_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i7300_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i7core_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/i82975x_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/ie31200_edac.ko 2023-06-22T17:38:29.916 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/sb_edac.ko 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/edac/x38_edac.ko 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/firmware 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/firmware/edd.ko 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/firmware/iscsi_ibft.ko 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/gpio 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/gpio/gpio-ich.ko 2023-06-22T17:38:29.917 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/gpio/gpio-viperboard.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-appleir.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-aureal.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-axff.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-dr.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-elecom.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-elo.ko 2023-06-22T17:38:29.918 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-emsff.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-gaff.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-gt683r.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-gyration.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-holtek-kbd.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-holtek-mouse.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-holtekff.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-hyperv.ko 2023-06-22T17:38:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-icade.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-keytouch.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-kye.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-lcpower.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-led.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-lenovo.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-logitech-dj.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-06-22T17:38:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-multitouch.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-ortek.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-penmount.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-petalynx.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-picolcd.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-pl.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-plantronics.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-primax.ko 2023-06-22T17:38:29.921 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-rmi.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-arvo.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-common.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-isku.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-kone.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-konepure.ko 2023-06-22T17:38:29.922 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-lua.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-pyra.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-ryos.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat-savu.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-roccat.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-saitek.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-samsung.ko 2023-06-22T17:38:29.923 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sensor-hub.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sjoy.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sony.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-speedlink.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-steelseries.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-sunplus.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-thrustmaster.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-tivo.ko 2023-06-22T17:38:29.924 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-tmff.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-topseed.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-twinhan.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-uclogic.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-waltop.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-wiimote.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-xinmo.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-zpff.ko 2023-06-22T17:38:29.925 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/hid-zydacron.ko 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/uhid.ko 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hid/wacom.ko 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv/hv_balloon.ko 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv/hv_utils.ko 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hv/hv_vmbus.ko 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon 2023-06-22T17:38:29.926 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/abituguru.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/abituguru3.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/acpi_power_meter.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ad7414.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ad7418.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adc128d818.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1025.ko 2023-06-22T17:38:29.927 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1026.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1029.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm1031.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adm9240.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ads7828.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7410.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7411.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7462.ko 2023-06-22T17:38:29.928 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7470.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7475.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/adt7x10.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/amc6821.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/applesmc.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/asb100.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/asc7621.ko 2023-06-22T17:38:29.929 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/asus_atk0110.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/atxp1.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/coretemp.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/dme1737.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ds1621.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ds620.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/emc1403.ko 2023-06-22T17:38:29.930 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/emc6w201.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/f71805f.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/f71882fg.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/f75375s.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/fam15h_power.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/fschmd.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/g760a.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/g762.ko 2023-06-22T17:38:29.931 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/gl518sm.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/gl520sm.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/hwmon-vid.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/i5500_temp.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/i5k_amb.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ibmaem.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ibmpex.ko 2023-06-22T17:38:29.932 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ina209.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ina2xx.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/it87.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/k10temp.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/k8temp.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lineage-pem.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm63.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm73.ko 2023-06-22T17:38:29.933 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm75.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm77.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm78.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm80.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm83.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm85.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm87.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm90.ko 2023-06-22T17:38:29.934 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm92.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm93.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm95234.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm95241.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/lm95245.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc2945.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4151.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4215.ko 2023-06-22T17:38:29.935 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4222.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4245.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4260.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/ltc4261.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max16065.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max1619.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max1668.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max197.ko 2023-06-22T17:38:29.936 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max6639.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max6650.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/max6697.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/mcp3021.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct6683.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct6775-core.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct6775.ko 2023-06-22T17:38:29.937 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/nct7802.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pc87360.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pc87427.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pcf8591.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-06-22T17:38:29.938 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/max16064.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/max34440.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/max8688.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-06-22T17:38:29.939 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/powr1220.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sch5627.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sch5636.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sch56xx-common.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sht15.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sht21.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/shtc1.ko 2023-06-22T17:38:29.940 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/sis5595.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/smsc47b397.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/smsc47m1.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/smsc47m192.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/thmc50.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp102.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp103.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp401.ko 2023-06-22T17:38:29.941 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/tmp421.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/via-cputemp.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/via686a.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/vt1211.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/vt8231.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83627ehf.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83627hf.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83781d.ko 2023-06-22T17:38:29.942 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83791d.ko 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83792d.ko 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83793.ko 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83795.ko 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83l785ts.ko 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/hwmon/w83l786ng.ko 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/algos 2023-06-22T17:38:29.943 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-06-22T17:38:29.944 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-i801.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-isch.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-06-22T17:38:29.945 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-parport.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-06-22T17:38:29.946 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-via.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-dev.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-mux.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-smbus.ko 2023-06-22T17:38:29.947 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/i2c/i2c-stub.ko 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_cm.ko 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_core.ko 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_umad.ko 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/iw_cm.ko 2023-06-22T17:38:29.948 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/rdma_cm.ko 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/cxgb4 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx4 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx5 2023-06-22T17:38:29.949 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mthca 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/ipoib 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/iser 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-06-22T17:38:29.950 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/isert 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srp 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srpt 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport 2023-06-22T17:38:29.951 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/fm801-gp.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/gameport.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/lightning.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/gameport/ns558.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/joydev.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/keyboard 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/keyboard/gpio_keys.ko 2023-06-22T17:38:29.952 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/appletouch.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/bcm5974.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/cyapatp.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/elan_i2c.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/sermouse.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/synaptics_usb.ko 2023-06-22T17:38:29.953 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/mouse/vsxxxaa.ko 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/rmi4 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/rmi4/rmi_core.ko 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/altera_ps2.ko 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/arc_ps2.ko 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/serio/serio_raw.ko 2023-06-22T17:38:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/input/sparse-keymap.ko 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/iommu 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/iommu/amd 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/iommu/amd/iommu_v2.ko 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-blinkm.ko 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-lm3530.ko 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-lp3944.ko 2023-06-22T17:38:29.955 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/leds-ss4200.ko 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-06-22T17:38:29.956 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/bcache 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/bcache/bcache.ko 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-bio-prison.ko 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-cache-smq.ko 2023-06-22T17:38:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-cache.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-crypt.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-delay.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-flakey.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-log-userspace.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-multipath.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-queue-length.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-raid.ko 2023-06-22T17:38:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-round-robin.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-service-time.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-switch.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-thin-pool.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/dm-verity.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/faulty.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/linear.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/multipath.ko 2023-06-22T17:38:29.959 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/persistent-data 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid0.ko 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid1.ko 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid10.ko 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/md/raid456.ko 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message 2023-06-22T17:38:29.960 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptbase.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptctl.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptfc.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptlan.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptsas.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptscsih.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/message/fusion/mptspi.ko 2023-06-22T17:38:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/lpc_ich.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/lpc_sch.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/mfd-core.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/sm501.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/viperboard.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/vx855.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/mfd/wl1273-core.ko 2023-06-22T17:38:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/altera-stapl 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/apds9802als.ko 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/apds990x.ko 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/bh1770glc.ko 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/cb710 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/cb710/cb710.ko 2023-06-22T17:38:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/at24.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/eeprom.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/eeprom/max6875.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/enclosure.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/hpilo.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/isl29003.ko 2023-06-22T17:38:29.964 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/isl29020.ko 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/lis3lv02d 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei/mei-me.ko 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei/mei-txe.ko 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/mei/mei.ko 2023-06-22T17:38:29.965 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-gru 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-gru/gru.ko 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp/xp.ko 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp/xpc.ko 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/tifm_7xx1.ko 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/tifm_core.ko 2023-06-22T17:38:29.966 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/tsl2550.ko 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/vmw_balloon.ko 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/vmw_vmci 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/bonding 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/bonding/bonding.ko 2023-06-22T17:38:29.967 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/b53 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/b53/b53_common.ko 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/bcm-sf2.ko 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/mv88e6060.ko 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/mv88e6xxx 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/dummy.ko 2023-06-22T17:38:29.968 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/eql.ko 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390 2023-06-22T17:38:29.969 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/8390.ko 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/adaptec 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/agere 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/agere/et131x.ko 2023-06-22T17:38:29.970 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/alteon 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/altera 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-06-22T17:38:29.971 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/alx 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1c 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1e 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atlx 2023-06-22T17:38:29.972 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-06-22T17:38:29.973 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/genet 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/brocade 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/brocade/bna 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio 2023-06-22T17:38:29.974 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-06-22T17:38:29.975 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/cisco 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/cisco/enic 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-06-22T17:38:29.976 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dlink 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-06-22T17:38:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/dnet.ko 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/emulex 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/emulex/benet 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/ethoc.ko 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/fealnx.ko 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e100.ko 2023-06-22T17:38:29.978 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000e 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/fm10k 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/i40e 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-06-22T17:38:29.979 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/iavf 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igb 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igbvf 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbe 2023-06-22T17:38:29.980 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbevf 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/jme.ko 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell/skge.ko 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-06-22T17:38:29.981 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx4 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx5 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/micrel 2023-06-22T17:38:29.982 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/myricom 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/myricom/myri10ge 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/natsemi 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/neterion 2023-06-22T17:38:29.983 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/nvidia 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/packetengines 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/netxen 2023-06-22T17:38:29.984 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rdc 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-06-22T17:38:29.985 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/atp.ko 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rocker 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sfc 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-06-22T17:38:29.986 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/silan 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sis 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sis/sis190.ko 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sis/sis900.ko 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-06-22T17:38:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro/stmmac 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/cassini.ko 2023-06-22T17:38:29.988 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/niu.ko 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/sungem.ko 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/tehuti 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/ti 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/ti/tlan.ko 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/via 2023-06-22T17:38:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/wiznet 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/xircom 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/geneve.ko 2023-06-22T17:38:29.990 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/hyperv 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ieee802154 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ieee802154/fakelb.ko 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ifb.ko 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ipvlan 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ipvlan/ipvlan.ko 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/macvlan.ko 2023-06-22T17:38:29.991 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/macvtap.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/mii.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/net_failover.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/netconsole.ko 2023-06-22T17:38:29.992 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/nlmon.ko 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/pcs 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/amd.ko 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-06-22T17:38:29.993 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm7xxx.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/bcm87xx.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/broadcom.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/cicada.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/davicom.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/dp83640.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/et1011c.ko 2023-06-22T17:38:29.994 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/icplus.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/lxt.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/marvell.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/micrel.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/national.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/phylink.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/qsemi.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/realtek.ko 2023-06-22T17:38:29.995 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/smsc.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/ste10Xp.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/phy/vitesse.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/bsd_comp.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_async.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_deflate.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_generic.ko 2023-06-22T17:38:29.996 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_mppe.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/ppp_synctty.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/pppoe.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/pppox.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/ppp/pptp.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/slip 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/slip/slhc.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/slip/slip.ko 2023-06-22T17:38:29.997 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/sungem_phy.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/tap.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_activebackup.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_broadcast.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_random.ko 2023-06-22T17:38:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/tun.ko 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/veth.ko 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/virtio_net.ko 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vmxnet3 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vxlan 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/vxlan/vxlan.ko 2023-06-22T17:38:29.999 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/xen-netback 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/xen-netback/xen-netback.ko 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/net/xen-netfront.ko 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ntb 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ntb/ntb.ko 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme/host 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme/host/nvme-core.ko 2023-06-22T17:38:30.000 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/nvme/host/nvme.ko 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport.ko 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport_cs.ko 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport_pc.ko 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/parport/parport_serial.ko 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci 2023-06-22T17:38:30.001 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/hotplug 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/pcie 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/pcie/aer_inject.ko 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pci/xen-pcifront.ko 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia/i82092.ko 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia/pd6729.ko 2023-06-22T17:38:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pcmcia/yenta_socket.ko 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pinctrl 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pinctrl/intel 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/acer-wmi.ko 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/acerhdf.ko 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/apple-gmux.ko 2023-06-22T17:38:30.003 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/asus-laptop.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/asus-wmi.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/classmate-laptop.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-06-22T17:38:30.004 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/hdaps.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/intel 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/intel_ips.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/msi-wmi.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/mxm-wmi.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-06-22T17:38:30.005 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/samsung-laptop.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/samsung-q10.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/topstar-laptop.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/toshiba_haps.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/wmi-bmof.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/platform/x86/wmi.ko 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/powercap 2023-06-22T17:38:30.006 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/powercap/intel_rapl_common.ko 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/powercap/intel_rapl_msr.ko 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients/pps-gpio.ko 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients/pps-ldisc.ko 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/clients/pps_parport.ko 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/pps/pps_core.ko 2023-06-22T17:38:30.007 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ptp 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ptp/ptp.ko 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ptp/ptp_kvm.ko 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-bq32k.ko 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-bq4802.ko 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1286.ko 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1307.ko 2023-06-22T17:38:30.008 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1374.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1511.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1553.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1672.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds1742.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds2404.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-ds3232.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-em3027.ko 2023-06-22T17:38:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-fm3130.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-isl12022.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-isl1208.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-m41t80.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-m48t35.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-m48t59.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-max6900.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-msm6242.ko 2023-06-22T17:38:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf2127.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf85063.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf8523.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf8563.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-pcf8583.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rp5c01.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rs5c372.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-06-22T17:38:30.011 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rx8025.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-rx8581.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/rtc/rtc-x1205.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/3w-9xxx.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/3w-sas.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/3w-xxxx.ko 2023-06-22T17:38:30.012 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/BusLogic.ko 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/a100u2w.ko 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aacraid 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aacraid/aacraid.ko 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/advansys.ko 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aic7xxx 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-06-22T17:38:30.013 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/am53c974.ko 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/arcmsr 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/atp870u.ko 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/be2iscsi 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bfa 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bfa/bfa.ko 2023-06-22T17:38:30.014 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2fc 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2i 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ch.ko 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/csiostor 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/csiostor/csiostor.ko 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi 2023-06-22T17:38:30.015 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb3i 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb4i 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/dc395x.ko 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-06-22T17:38:30.016 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/dmx3191d.ko 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/esas2r 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/esas2r/esas2r.ko 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/esp_scsi.ko 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fcoe 2023-06-22T17:38:30.017 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fcoe/fcoe.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fnic 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/fnic/fnic.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/hpsa.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/hptiop.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/hv_storvsc.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/initio.ko 2023-06-22T17:38:30.018 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ipr.ko 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ips.ko 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/isci 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/isci/isci.ko 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/iscsi_tcp.ko 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libfc 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libfc/libfc.ko 2023-06-22T17:38:30.019 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libiscsi.ko 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libiscsi_tcp.ko 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libsas 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/libsas/libsas.ko 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/lpfc 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/lpfc/lpfc.ko 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid.ko 2023-06-22T17:38:30.020 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mpt3sas 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mvsas 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mvsas/mvsas.ko 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/mvumi.ko 2023-06-22T17:38:30.021 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/pm8001 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/pmcraid.ko 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla1280.ko 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla2xxx 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla4xxx 2023-06-22T17:38:30.022 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/raid_class.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_debug.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_fc.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_sas.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_spi.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/scsi_transport_srp.ko 2023-06-22T17:38:30.023 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/ses.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/st.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/stex.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/sym53c8xx_2 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/virtio_scsi.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/vmw_pvscsi.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/scsi/wd719x.ko 2023-06-22T17:38:30.024 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ssb 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ssb/ssb.ko 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/iscsi 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/loopback 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/loopback/tcm_loop.ko 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_file.ko 2023-06-22T17:38:30.025 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_iblock.ko 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_mod.ko 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_pscsi.ko 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/target_core_user.ko 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/tcm_fc 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel 2023-06-22T17:38:30.026 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-06-22T17:38:30.027 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-06-22T17:38:30.028 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/ipwireless 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/n_gsm.ko 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/n_hdlc.ko 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/nozomi.ko 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/8250 2023-06-22T17:38:30.029 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/arc_uart.ko 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/jsm 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/serial/jsm/jsm.ko 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/tty/synclink_gt.ko 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/core 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/core/ufshcd-core.ko 2023-06-22T17:38:30.030 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/host 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio.ko 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_aec.ko 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_cif.ko 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_pci_generic.ko 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/uio/uio_sercos3.ko 2023-06-22T17:38:30.031 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/cdc-acm.ko 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/cdc-wdm.ko 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/usblp.ko 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/class/usbtmc.ko 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/host 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/host/sl811-hcd.ko 2023-06-22T17:38:30.032 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/image 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/image/mdc800.ko 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/image/microtek.ko 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/adutux.ko 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/appledisplay.ko 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/emi26.ko 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/emi62.ko 2023-06-22T17:38:30.033 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/ezusb.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/idmouse.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/iowarrior.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/isight_firmware.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/ldusb.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/legousbtower.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/sisusbvga 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-06-22T17:38:30.034 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/trancevibrator.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/usb3503.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/usblcd.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/usbsevseg.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/uss720.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/misc/yurex.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/phy 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/phy/phy-generic.ko 2023-06-22T17:38:30.035 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/aircable.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ark3116.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/belkin_sa.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ch341.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/cp210x.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/cyberjack.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/cypress_m8.ko 2023-06-22T17:38:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/digi_acceleport.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/empeg.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ftdi_sio.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/garmin_gps.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/io_edgeport.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/io_ti.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ipaq.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ipw.ko 2023-06-22T17:38:30.037 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ir-usb.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/keyspan.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/keyspan_pda.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/kl5kusb105.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/kobil_sct.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/mct_u232.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/mos7720.ko 2023-06-22T17:38:30.038 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/mos7840.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/navman.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/omninet.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/opticon.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/option.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/oti6858.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/pl2303.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/qcaux.ko 2023-06-22T17:38:30.039 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/qcserial.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/quatech2.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/safe_serial.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/sierra.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/spcp8x5.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ssu100.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/symbolserial.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-06-22T17:38:30.040 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/usb_debug.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/usb_wwan.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/visor.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/whiteheat.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/serial/xsens_mt.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/uas.ko 2023-06-22T17:38:30.041 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-alauda.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-cypress.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-datafab.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-freecom.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-isd200.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-karma.ko 2023-06-22T17:38:30.042 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-onetouch.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-realtek.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-sddr09.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-sddr55.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/ums-usbat.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/storage/usb-storage.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip/usbip-core.ko 2023-06-22T17:38:30.043 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip/usbip-host.ko 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/pci 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/pci/vfio-pci.ko 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/vfio.ko 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-06-22T17:38:30.044 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost.ko 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost_iotlb.ko 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost_net.ko 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/vhost/vhost_scsi.ko 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight/apple_bl.ko 2023-06-22T17:38:30.045 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight/lcd.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/video/backlight/platform_lcd.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_balloon.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_mmio.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_pci.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-06-22T17:38:30.046 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/virtio/virtio_ring.ko 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/masters 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/masters/ds2482.ko 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/masters/ds2490.ko 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-06-22T17:38:30.047 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-06-22T17:38:30.048 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_smem.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/slaves/w1_therm.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/w1/wire.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/alim1535_wdt.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/alim7101_wdt.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/f71808e_wdt.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/hpwdt.ko 2023-06-22T17:38:30.049 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/i6300esb.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/iTCO_wdt.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/ib700wdt.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/ibmasr.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/it8712f_wdt.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/it87_wdt.ko 2023-06-22T17:38:30.050 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/machzwd.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/nv_tco.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/pcwd_pci.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/pcwd_usb.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/sch311x_wdt.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/softdog.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/sp5100_tco.ko 2023-06-22T17:38:30.051 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/via_wdt.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/w83877f_wdt.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/w83977f_wdt.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/wdt_pci.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/watchdog/xen_wdt.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-acpi-processor.ko 2023-06-22T17:38:30.052 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-evtchn.ko 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-gntalloc.ko 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-gntdev.ko 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-pciback 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xen-privcmd.ko 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xenfs 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/drivers/xen/xenfs/xenfs.ko 2023-06-22T17:38:30.053 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/9p 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/9p/9p.ko 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/affs 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/affs/affs.ko 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/befs 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/befs/befs.ko 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/binfmt_misc.ko 2023-06-22T17:38:30.054 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/btrfs 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/btrfs/btrfs.ko 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cachefiles 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cachefiles/cachefiles.ko 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ceph 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ceph/ceph.ko 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/coda 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/coda/coda.ko 2023-06-22T17:38:30.055 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cramfs 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/cramfs/cramfs.ko 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/dlm 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/dlm/dlm.ko 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ecryptfs 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ecryptfs/ecryptfs.ko 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/f2fs 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/f2fs/f2fs.ko 2023-06-22T17:38:30.056 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat/fat.ko 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat/msdos.ko 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fat/vfat.ko 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fscache 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fscache/fscache.ko 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fuse 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fuse/cuse.ko 2023-06-22T17:38:30.057 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/fuse/fuse.ko 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/gfs2 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/gfs2/gfs2.ko 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfs 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfs/hfs.ko 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfsplus 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/hfsplus/hfsplus.ko 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/isofs 2023-06-22T17:38:30.058 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/isofs/isofs.ko 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/jfs 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/jfs/jfs.ko 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/lockd 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/lockd/lockd.ko 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/minix 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/minix/minix.ko 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/netfs 2023-06-22T17:38:30.059 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/netfs/netfs.ko 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/blocklayout 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/filelayout 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/flexfilelayout 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-06-22T17:38:30.060 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/nfs.ko 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/nfsv3.ko 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs/nfsv4.ko 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs_common 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs_common/grace.ko 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfs_common/nfs_acl.ko 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfsd 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nfsd/nfsd.ko 2023-06-22T17:38:30.061 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nilfs2 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nilfs2/nilfs2.ko 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-celtic.ko 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-centeuro.ko 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-croatian.ko 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-cyrillic.ko 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-gaelic.ko 2023-06-22T17:38:30.062 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-greek.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-iceland.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-inuit.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-roman.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-romanian.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/mac-turkish.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp1250.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp1251.ko 2023-06-22T17:38:30.063 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp1255.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp737.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp775.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp850.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp852.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp855.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp857.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp860.ko 2023-06-22T17:38:30.064 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp861.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp862.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp863.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp864.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp865.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp866.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp869.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp874.ko 2023-06-22T17:38:30.065 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp932.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp936.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp949.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_cp950.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_euc-jp.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-1.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-13.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-14.ko 2023-06-22T17:38:30.066 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-15.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-2.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-3.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-4.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-5.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-6.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-7.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_iso8859-9.ko 2023-06-22T17:38:30.067 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_koi8-r.ko 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_koi8-ru.ko 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_koi8-u.ko 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/nls/nls_utf8.ko 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/cluster 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlm 2023-06-22T17:38:30.068 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlmfs 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2.ko 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/overlayfs 2023-06-22T17:38:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/overlayfs/overlay.ko 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/pstore 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/pstore/ramoops.ko 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/reiserfs 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/reiserfs/reiserfs.ko 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/romfs 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/romfs/romfs.ko 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb 2023-06-22T17:38:30.070 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/client 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/client/cifs.ko 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/common 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/common/cifs_arc4.ko 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/smb/common/cifs_md4.ko 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/squashfs 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/squashfs/squashfs.ko 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/sysv 2023-06-22T17:38:30.071 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/sysv/sysv.ko 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/udf 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/udf/udf.ko 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ufs 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/ufs/ufs.ko 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/xfs 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/fs/xfs/xfs.ko 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel 2023-06-22T17:38:30.072 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/rcu 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/rcu/rcutorture.ko 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/torture.ko 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/trace 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/asn1_encoder.ko 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crc-itu-t.ko 2023-06-22T17:38:30.073 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crc8.ko 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crypto 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crypto/libarc4.ko 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/crypto/libdes.ko 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/libcrc32c.ko 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lru_cache.ko 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lz4 2023-06-22T17:38:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lz4/lz4_compress.ko 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/lz4/lz4hc_compress.ko 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/math 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/math/cordic.ko 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/raid6 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/raid6/raid6_pq.ko 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/reed_solomon 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/reed_solomon/reed_solomon.ko 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/ts_bm.ko 2023-06-22T17:38:30.075 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/ts_fsm.ko 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/ts_kmp.ko 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/zstd 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/lib/zstd/zstd_compress.ko 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/mm 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/mm/hwpoison-inject.ko 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net 2023-06-22T17:38:30.076 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/6lowpan.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_dest.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_fragment.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_hop.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_ipv6.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_mobility.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_routing.ko 2023-06-22T17:38:30.077 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/6lowpan/nhc_udp.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/garp.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/mrp.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/p8022.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/psnap.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/802/stp.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/8021q 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/8021q/8021q.ko 2023-06-22T17:38:30.078 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet.ko 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet_fd.ko 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet_rdma.ko 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/9p/9pnet_virtio.ko 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/batman-adv 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/batman-adv/batman-adv.ko 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge 2023-06-22T17:38:30.079 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/br_netfilter.ko 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/bridge.ko 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_among.ko 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_arp.ko 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-06-22T17:38:30.080 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_ip.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_limit.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_log.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_mark.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-06-22T17:38:30.081 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_snat.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_stp.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-06-22T17:38:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/ebtables.ko 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ceph 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ceph/libceph.ko 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/core 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/core/failover.ko 2023-06-22T17:38:30.083 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/core/pktgen.ko 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp.ko 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp_diag.ko 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp_ipv4.ko 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dccp/dccp_ipv6.ko 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dns_resolver 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dns_resolver/dns_resolver.ko 2023-06-22T17:38:30.084 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/dsa_core.ko 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_brcm.ko 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_dsa.ko 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_none.ko 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/dsa/tag_trailer.ko 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/6lowpan 2023-06-22T17:38:30.085 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/ieee802154.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ieee802154/ieee802154_socket.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ah4.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/esp4.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/fou.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/gre.ko 2023-06-22T17:38:30.086 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/inet_diag.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ip_gre.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ip_tunnel.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ip_vti.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ipcomp.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/ipip.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/arp_tables.ko 2023-06-22T17:38:30.087 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-06-22T17:38:30.088 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/iptable_security.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-06-22T17:38:30.089 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_bic.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_dctcp.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_diag.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_highspeed.ko 2023-06-22T17:38:30.090 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_htcp.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_hybla.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_illinois.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_lp.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_scalable.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_vegas.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_veno.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_westwood.ko 2023-06-22T17:38:30.091 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tcp_yeah.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/tunnel4.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/udp_diag.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/udp_tunnel.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv4/xfrm4_tunnel.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ah6.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/esp6.ko 2023-06-22T17:38:30.092 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/fou6.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ip6_tunnel.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ip6_vti.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/ipcomp6.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-06-22T17:38:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-06-22T17:38:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-06-22T17:38:30.095 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/sit.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/tunnel6.ko 2023-06-22T17:38:30.096 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/ipv6/xfrm6_tunnel.ko 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/key 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/key/af_key.ko 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_core.ko 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_debugfs.ko 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_eth.ko 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_ip.ko 2023-06-22T17:38:30.097 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_ip6.ko 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_netlink.ko 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/l2tp/l2tp_ppp.ko 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/llc 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/llc/llc.ko 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mac802154 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mac802154/mac802154.ko 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mpls 2023-06-22T17:38:30.098 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/mpls/mpls_gso.ko 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set.ko 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-06-22T17:38:30.099 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-06-22T17:38:30.100 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-06-22T17:38:30.101 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-06-22T17:38:30.102 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conncount.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_h323.ko 2023-06-22T17:38:30.103 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_irc.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_sane.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_sip.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-06-22T17:38:30.104 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_dup_netdev.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_flow_table.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_flow_table_inet.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_log_syslog.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_amanda.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_ftp.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_irc.ko 2023-06-22T17:38:30.105 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_sip.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_nat_tftp.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_synproxy_core.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nf_tables.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_acct.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_log.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_osf.ko 2023-06-22T17:38:30.106 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nfnetlink_queue.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_chain_nat.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_compat.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_ct.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_dup_netdev.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_fib.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_fwd_netdev.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_hash.ko 2023-06-22T17:38:30.107 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_limit.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_log.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_masq.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_nat.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_numgen.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_queue.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_quota.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_redir.ko 2023-06-22T17:38:30.108 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_reject.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_reject_inet.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_socket.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/nft_tunnel.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_AUDIT.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CHECKSUM.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CLASSIFY.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-06-22T17:38:30.109 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_CT.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_DSCP.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_HL.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_HMARK.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_IDLETIMER.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_LED.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_LOG.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_MASQUERADE.ko 2023-06-22T17:38:30.110 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_NETMAP.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_NFLOG.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_NFQUEUE.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_RATEEST.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_REDIRECT.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_SECMARK.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TCPMSS.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-06-22T17:38:30.111 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TEE.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TPROXY.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_TRACE.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_addrtype.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_bpf.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_cgroup.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_cluster.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_comment.ko 2023-06-22T17:38:30.112 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connbytes.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connlabel.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connlimit.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_connmark.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_conntrack.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_cpu.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_dccp.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_devgroup.ko 2023-06-22T17:38:30.113 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_dscp.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_ecn.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_esp.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_hashlimit.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_helper.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_hl.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_ipcomp.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_iprange.ko 2023-06-22T17:38:30.114 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_ipvs.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_l2tp.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_length.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_limit.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_mac.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_mark.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_multiport.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_nat.ko 2023-06-22T17:38:30.115 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_nfacct.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_osf.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_owner.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_physdev.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_pkttype.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_policy.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_quota.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_rateest.ko 2023-06-22T17:38:30.116 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_realm.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_recent.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_sctp.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_set.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_socket.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_state.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_statistic.ko 2023-06-22T17:38:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_string.ko 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_tcpmss.ko 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_time.ko 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netfilter/xt_u32.ko 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netlink 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/netlink/netlink_diag.ko 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/nsh 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/nsh/nsh.ko 2023-06-22T17:38:30.118 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/openvswitch.ko 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/vport-geneve.ko 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/vport-gre.ko 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/openvswitch/vport-vxlan.ko 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/packet 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/packet/af_packet_diag.ko 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds 2023-06-22T17:38:30.119 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds/rds.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds/rds_rdma.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/rds/rds_tcp.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_csum.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_gact.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_ipt.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_mirred.ko 2023-06-22T17:38:30.120 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_nat.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_pedit.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_police.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_simple.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_skbedit.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/act_vlan.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_basic.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_bpf.ko 2023-06-22T17:38:30.121 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_flow.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_fw.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_route.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/cls_u32.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_cmp.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_ipset.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_meta.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_nbyte.ko 2023-06-22T17:38:30.122 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_text.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/em_u32.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_choke.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_codel.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_drr.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_fq.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_gred.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_hfsc.ko 2023-06-22T17:38:30.123 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_hhf.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_htb.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_ingress.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_mqprio.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_mqprio_lib.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_multiq.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_netem.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_pie.ko 2023-06-22T17:38:30.124 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_plug.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_prio.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_qfq.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_red.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_sfb.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_sfq.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_tbf.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sched/sch_teql.ko 2023-06-22T17:38:30.125 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sctp 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sctp/sctp.ko 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sctp/sctp_diag.ko 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/auth_gss 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/sunrpc.ko 2023-06-22T17:38:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/xprtrdma 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/tipc 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/tipc/diag.ko 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/tipc/tipc.ko 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/unix 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/unix/unix_diag.ko 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock 2023-06-22T17:38:30.127 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vsock.ko 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vsock_diag.ko 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/vmw_vsock/vsock_loopback.ko 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/xfrm 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/net/xfrm/xfrm_ipcomp.ko 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security 2023-06-22T17:38:30.128 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/encrypted-keys 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/trusted-keys 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/security/keys/trusted-keys/trusted.ko 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/virt 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/virt/lib 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/kernel/virt/lib/irqbypass.ko 2023-06-22T17:38:30.129 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.alias 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.alias.bin 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.builtin 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.builtin.bin 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.builtin.modinfo 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.dep 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.dep.bin 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.devname 2023-06-22T17:38:30.130 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.order 2023-06-22T17:38:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.softdep 2023-06-22T17:38:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.symbols 2023-06-22T17:38:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.4.0-rc5-g1c622d92b6f1/modules.symbols.bin 2023-06-22T17:38:30.132 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:30.132 DEBUG:teuthology.orchestra.run.smithi039:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-06-22T17:38:30.157 INFO:teuthology.orchestra.run.smithi039.stdout:postinstall scriptlet (using /bin/sh): 2023-06-22T17:38:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 -a -r /boot/System.map-6.4.0-rc5-g1c622d92b6f1 ]; then 2023-06-22T17:38:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:cp /boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 /boot/.vmlinuz-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-22T17:38:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:cp /boot/System.map-6.4.0-rc5-g1c622d92b6f1 /boot/.System.map-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-22T17:38:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:rm -f /boot/vmlinuz-6.4.0-rc5-g1c622d92b6f1 /boot/System.map-6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/installkernel 6.4.0-rc5-g1c622d92b6f1 /boot/.vmlinuz-6.4.0-rc5-g1c622d92b6f1-rpm /boot/.System.map-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-22T17:38:30.158 INFO:teuthology.orchestra.run.smithi039.stdout:rm -f /boot/.vmlinuz-6.4.0-rc5-g1c622d92b6f1-rpm /boot/.System.map-6.4.0-rc5-g1c622d92b6f1-rpm 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:preuninstall scriptlet (using /bin/sh): 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:new-kernel-pkg --remove 6.4.0-rc5-g1c622d92b6f1 --rminitrd --initrdfile=/boot/initramfs-6.4.0-rc5-g1c622d92b6f1.img 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-install remove 6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2023-06-22T17:38:30.159 INFO:teuthology.orchestra.run.smithi039.stdout:postuninstall scriptlet (using /bin/sh): 2023-06-22T17:38:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /sbin/update-bootloader ]; then 2023-06-22T17:38:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/update-bootloader --remove 6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:30.160 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2023-06-22T17:38:30.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -qi grub2-tools 2023-06-22T17:38:30.239 INFO:teuthology.orchestra.run.smithi039.stdout:Name : grub2-tools 2023-06-22T17:38:30.240 INFO:teuthology.orchestra.run.smithi039.stdout:Epoch : 1 2023-06-22T17:38:30.240 INFO:teuthology.orchestra.run.smithi039.stdout:Version : 2.02 2023-06-22T17:38:30.240 INFO:teuthology.orchestra.run.smithi039.stdout:Release : 99.el8_4.1 2023-06-22T17:38:30.241 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: x86_64 2023-06-22T17:38:30.241 INFO:teuthology.orchestra.run.smithi039.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-22T17:38:30.241 INFO:teuthology.orchestra.run.smithi039.stdout:Group : System Environment/Base 2023-06-22T17:38:30.242 INFO:teuthology.orchestra.run.smithi039.stdout:Size : 9264631 2023-06-22T17:38:30.242 INFO:teuthology.orchestra.run.smithi039.stdout:License : GPLv3+ 2023-06-22T17:38:30.242 INFO:teuthology.orchestra.run.smithi039.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-22T17:38:30.243 INFO:teuthology.orchestra.run.smithi039.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-22T17:38:30.243 INFO:teuthology.orchestra.run.smithi039.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-22T17:38:30.243 INFO:teuthology.orchestra.run.smithi039.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-22T17:38:30.243 INFO:teuthology.orchestra.run.smithi039.stdout:Relocations : (not relocatable) 2023-06-22T17:38:30.244 INFO:teuthology.orchestra.run.smithi039.stdout:Packager : Red Hat, Inc. 2023-06-22T17:38:30.244 INFO:teuthology.orchestra.run.smithi039.stdout:Vendor : Red Hat, Inc. 2023-06-22T17:38:30.244 INFO:teuthology.orchestra.run.smithi039.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-22T17:38:30.244 INFO:teuthology.orchestra.run.smithi039.stdout:Summary : Support tools for GRUB. 2023-06-22T17:38:30.245 INFO:teuthology.orchestra.run.smithi039.stdout:Description : 2023-06-22T17:38:30.245 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:38:30.245 INFO:teuthology.orchestra.run.smithi039.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-22T17:38:30.246 INFO:teuthology.orchestra.run.smithi039.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-22T17:38:30.246 INFO:teuthology.orchestra.run.smithi039.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-22T17:38:30.247 INFO:teuthology.orchestra.run.smithi039.stdout:hardware devices. 2023-06-22T17:38:30.247 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:38:30.247 INFO:teuthology.orchestra.run.smithi039.stdout:This subpackage provides tools for support of all platforms. 2023-06-22T17:38:30.248 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-22T17:38:30.248 INFO:teuthology.task.kernel:Updating grub on smithi039 to boot 6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:30.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-22T17:38:35.819 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2023-06-22T17:38:37.601 INFO:teuthology.orchestra.run.smithi039.stderr:done 2023-06-22T17:38:37.603 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2023-06-22T17:38:37.618 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.hkHr9wphSO 2023-06-22T17:38:37.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hkHr9wphSO 2023-06-22T17:38:37.687 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.hkHr9wphSO 2023-06-22T17:38:37.847 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.hkHr9wphSO is 6KB 2023-06-22T17:38:37.897 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.hkHr9wphSO 2023-06-22T17:38:37.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo /bin/ls /boot/loader/entries || true 2023-06-22T17:38:37.981 INFO:teuthology.orchestra.run.smithi039.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-22T17:38:37.982 INFO:teuthology.orchestra.run.smithi039.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-22T17:38:37.982 INFO:teuthology.orchestra.run.smithi039.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-22T17:38:37.982 INFO:teuthology.orchestra.run.smithi039.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-22T17:38:37.982 INFO:teuthology.orchestra.run.smithi039.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.4.0-rc5-g1c622d92b6f1.conf 2023-06-22T17:38:37.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:38:38.121 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2023-06-22T17:39:08.124 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-06-22T17:39:08.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-06-22T17:39:08.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:39:42.576 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2023-06-22T17:40:12.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-06-22T17:40:12.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:41:12.672 DEBUG:teuthology.orchestra.remote:timed out 2023-06-22T17:41:42.674 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-06-22T17:41:42.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:41:42.988 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-06-22T17:41:43.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-06-22T17:41:43.349 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.4.0-rc5-g1c622d92b6f1"... 2023-06-22T17:41:43.350 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2023-06-22T17:41:43.385 INFO:teuthology.orchestra.run.smithi039.stdout:6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:41:43.385 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-rc5-g1c622d92b6f1 vs 6.4.0-rc5-g1c622d92b6f1 2023-06-22T17:41:43.386 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-22T17:41:43.386 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-22T17:41:44.387 DEBUG:teuthology.parallel:result is None 2023-06-22T17:41:44.387 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-22T17:41:44.395 INFO:teuthology.task.internal:Creating test directory... 2023-06-22T17:41:44.396 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-22T17:41:44.400 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-22T17:41:44.403 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-22T17:41:44.435 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-22T17:41:44.442 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-22T17:41:44.484 INFO:teuthology.task.internal:Creating archive directory... 2023-06-22T17:41:44.484 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-22T17:41:44.489 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-22T17:41:44.491 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-22T17:41:44.551 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-22T17:41:44.559 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-22T17:41:44.559 DEBUG:teuthology.orchestra.run.smithi101:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-22T17:41:44.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:41:44.579 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-22T17:41:44.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:41:44.595 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-22T17:41:44.599 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-22T17:41:44.621 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-22T17:41:44.663 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T17:41:44.684 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T17:41:44.693 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T17:41:44.729 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T17:41:44.733 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T17:41:44.768 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-22T17:41:44.771 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-22T17:41:44.779 INFO:teuthology.task.internal:Configuring sudo... 2023-06-22T17:41:44.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-22T17:41:44.784 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-22T17:41:44.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-22T17:41:44.843 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-22T17:41:44.852 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-22T17:41:44.853 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-22T17:41:44.857 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-22T17:41:44.859 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-22T17:41:44.888 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T17:41:44.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T17:41:45.085 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T17:41:45.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T17:41:45.226 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T17:41:45.226 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-22T17:41:45.288 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T17:41:45.335 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T17:41:45.376 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T17:41:45.464 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T17:41:45.501 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T17:41:45.502 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-22T17:41:45.568 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T17:41:45.612 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-22T17:41:45.656 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T17:41:45.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-22T17:41:45.781 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T17:41:45.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-22T17:41:45.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2023-06-22T17:41:45.850 DEBUG:teuthology.orchestra.run.smithi101:> sudo service rsyslog restart 2023-06-22T17:41:45.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2023-06-22T17:41:45.915 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T17:41:45.947 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T17:41:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T17:41:46.368 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-22T17:41:46.376 INFO:teuthology.task.internal:Starting timer... 2023-06-22T17:41:46.376 INFO:teuthology.run_tasks:Running task pcp... 2023-06-22T17:41:46.424 INFO:teuthology.run_tasks:Running task selinux... 2023-06-22T17:41:46.470 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-22T17:41:46.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2023-06-22T17:41:46.589 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: [ OK ] 2023-06-22T17:41:46.592 DEBUG:teuthology.orchestra.run.smithi101:> sudo service auditd rotate 2023-06-22T17:41:46.658 INFO:teuthology.orchestra.run.smithi101.stdout:Rotating logs: [ OK ] 2023-06-22T17:41:46.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2023-06-22T17:41:46.716 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: [ OK ] 2023-06-22T17:41:46.718 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-22T17:41:46.719 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2023-06-22T17:41:46.765 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2023-06-22T17:41:46.765 DEBUG:teuthology.orchestra.run.smithi101:> /usr/sbin/getenforce 2023-06-22T17:41:46.787 INFO:teuthology.orchestra.run.smithi101.stdout:Permissive 2023-06-22T17:41:46.788 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2023-06-22T17:41:46.826 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2023-06-22T17:41:46.826 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi101.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2023-06-22T17:41:46.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T17:41:46.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:41:46.891 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T17:41:46.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:41:46.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T17:41:46.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T17:41:46.941 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-22T17:41:46.941 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2023-06-22T17:41:47.004 DEBUG:teuthology.orchestra.run.smithi101:> sudo /usr/sbin/setenforce permissive 2023-06-22T17:41:47.028 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2023-06-22T17:41:47.064 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-22T17:41:47.074 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-22T17:41:47.075 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-22T17:41:47.096 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-22T17:41:47.097 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi101.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-22T17:58:54.903 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi101.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2023-06-22T17:58:54.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-06-22T17:58:54.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:54.992 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-06-22T17:58:55.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-06-22T17:58:55.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:55.161 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-06-22T17:58:55.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-06-22T17:58:55.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:55.297 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-06-22T17:58:55.365 INFO:teuthology.run_tasks:Running task clock... 2023-06-22T17:58:55.376 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-22T17:58:55.377 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-22T17:58:55.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:58:55.382 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-22T17:58:55.382 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:58:55.384 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-22T17:58:55.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T17:58:55.412 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-22T17:58:55.426 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-22T17:58:55.436 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-22T17:58:55.445 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-22T17:58:55.451 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-22T17:58:55.460 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: ntpd: command not found 2023-06-22T17:58:55.471 INFO:teuthology.orchestra.run.smithi101.stdout:506 Cannot talk to daemon 2023-06-22T17:58:55.481 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-22T17:58:55.483 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2023-06-22T17:58:55.484 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-22T17:58:55.493 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2023-06-22T17:58:55.498 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-22T17:58:55.508 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-22T17:58:55.521 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-22T17:58:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2023-06-22T17:58:55.568 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2023-06-22T17:58:55.571 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-06-22T17:58:55.573 INFO:teuthology.orchestra.run.smithi101.stdout:210 Number of sources = 4 2023-06-22T17:58:55.573 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:58:55.574 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-06-22T17:58:55.574 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.574 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.574 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.574 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.595 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-06-22T17:58:55.598 INFO:teuthology.orchestra.run.smithi133.stdout:210 Number of sources = 4 2023-06-22T17:58:55.598 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:58:55.598 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-06-22T17:58:55.598 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.599 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.599 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.599 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-22T17:58:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-22T17:58:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2023-06-22T17:58:55.729 INFO:teuthology.orchestra.run.smithi039.stdout:210 Number of sources = 4 2023-06-22T17:58:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T17:58:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2023-06-22T17:58:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.731 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T17:58:55.732 INFO:teuthology.run_tasks:Running task pexec... 2023-06-22T17:58:55.742 INFO:teuthology.task.pexec:Executing custom commands... 2023-06-22T17:58:55.743 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi039.front.sepia.ceph.com 2023-06-22T17:58:55.743 DEBUG:teuthology.orchestra.run.smithi039:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-22T17:58:55.744 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi101.front.sepia.ceph.com 2023-06-22T17:58:55.744 DEBUG:teuthology.orchestra.run.smithi101:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-22T17:58:55.745 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi133.front.sepia.ceph.com 2023-06-22T17:58:55.745 DEBUG:teuthology.orchestra.run.smithi133:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-22T17:58:56.129 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-06-22T17:58:56.130 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-06-22T17:58:56.279 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-06-22T17:58:56.922 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:10:46 ago on Thu 22 Jun 2023 05:48:10 PM UTC. 2023-06-22T17:58:56.924 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:10:42 ago on Thu 22 Jun 2023 05:48:14 PM UTC. 2023-06-22T17:58:57.261 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:47 ago on Thu 22 Jun 2023 05:57:09 PM UTC. 2023-06-22T17:58:58.961 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-06-22T17:58:58.962 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout:Resetting modules: 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-06-22T17:58:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:58:58.976 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-06-22T17:58:58.977 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout: Package Architecture Version Repository Size 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout:Resetting modules: 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-06-22T17:58:58.978 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-06-22T17:58:58.979 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:58:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-06-22T17:58:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2023-06-22T17:58:59.495 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2023-06-22T17:58:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2023-06-22T17:58:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:Resetting modules: 2023-06-22T17:58:59.495 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools 2023-06-22T17:58:59.496 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:58:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2023-06-22T17:58:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2023-06-22T17:58:59.496 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:58:59.804 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:10:45 ago on Thu 22 Jun 2023 05:48:14 PM UTC. 2023-06-22T17:58:59.817 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:10:49 ago on Thu 22 Jun 2023 05:48:10 PM UTC. 2023-06-22T17:59:00.679 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:51 ago on Thu 22 Jun 2023 05:57:09 PM UTC. 2023-06-22T17:59:01.839 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-06-22T17:59:01.853 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-06-22T17:59:02.287 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:02.301 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:02.573 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-06-22T17:59:03.046 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:10:52 ago on Thu 22 Jun 2023 05:48:10 PM UTC. 2023-06-22T17:59:03.068 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:10:48 ago on Thu 22 Jun 2023 05:48:14 PM UTC. 2023-06-22T17:59:03.164 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:04.160 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:54 ago on Thu 22 Jun 2023 05:57:09 PM UTC. 2023-06-22T17:59:05.066 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-06-22T17:59:05.080 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-06-22T17:59:05.115 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================================= 2023-06-22T17:59:05.115 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-06-22T17:59:05.115 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================================= 2023-06-22T17:59:05.116 INFO:teuthology.orchestra.run.smithi133.stdout:Installing group/module packages: 2023-06-22T17:59:05.116 INFO:teuthology.orchestra.run.smithi133.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-06-22T17:59:05.116 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-06-22T17:59:05.116 INFO:teuthology.orchestra.run.smithi133.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-06-22T17:59:05.116 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-22T17:59:05.117 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-06-22T17:59:05.117 INFO:teuthology.orchestra.run.smithi133.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-06-22T17:59:05.117 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-06-22T17:59:05.117 INFO:teuthology.orchestra.run.smithi133.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-22T17:59:05.117 INFO:teuthology.orchestra.run.smithi133.stdout:Installing module profiles: 2023-06-22T17:59:05.117 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools/common 2023-06-22T17:59:05.118 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:05.118 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-06-22T17:59:05.118 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools rhel8 2023-06-22T17:59:05.118 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:05.118 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-06-22T17:59:05.119 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================================= 2023-06-22T17:59:05.119 INFO:teuthology.orchestra.run.smithi133.stdout:Install 7 Packages 2023-06-22T17:59:05.119 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:05.120 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 16 M 2023-06-22T17:59:05.120 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 57 M 2023-06-22T17:59:05.120 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-06-22T17:59:05.129 INFO:teuthology.orchestra.run.smithi101.stdout:========================================================================================================= 2023-06-22T17:59:05.130 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-06-22T17:59:05.130 INFO:teuthology.orchestra.run.smithi101.stdout:========================================================================================================= 2023-06-22T17:59:05.130 INFO:teuthology.orchestra.run.smithi101.stdout:Installing group/module packages: 2023-06-22T17:59:05.130 INFO:teuthology.orchestra.run.smithi101.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-06-22T17:59:05.130 INFO:teuthology.orchestra.run.smithi101.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-06-22T17:59:05.130 INFO:teuthology.orchestra.run.smithi101.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout:Installing module profiles: 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools/common 2023-06-22T17:59:05.131 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling module streams: 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools rhel8 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout:========================================================================================================= 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout:Install 7 Packages 2023-06-22T17:59:05.132 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:05.133 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 16 M 2023-06-22T17:59:05.133 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 57 M 2023-06-22T17:59:05.133 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-06-22T17:59:05.180 INFO:teuthology.orchestra.run.smithi133.stdout:(1/7): cockpit-podman-32-2.module+el8.4.0+11990 6.4 MB/s | 410 kB 00:00 2023-06-22T17:59:05.189 INFO:teuthology.orchestra.run.smithi101.stdout:(1/7): yajl-2.1.0-10.el8.x86_64.rpm 698 kB/s | 41 kB 00:00 2023-06-22T17:59:05.192 INFO:teuthology.orchestra.run.smithi133.stdout:(2/7): yajl-2.1.0-10.el8.x86_64.rpm 539 kB/s | 41 kB 00:00 2023-06-22T17:59:05.196 INFO:teuthology.orchestra.run.smithi133.stdout:(3/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 2.4 MB/s | 192 kB 00:00 2023-06-22T17:59:05.201 INFO:teuthology.orchestra.run.smithi101.stdout:(2/7): cockpit-podman-32-2.module+el8.4.0+11990 5.7 MB/s | 410 kB 00:00 2023-06-22T17:59:05.203 INFO:teuthology.orchestra.run.smithi101.stdout:(3/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 2.6 MB/s | 192 kB 00:00 2023-06-22T17:59:05.220 INFO:teuthology.orchestra.run.smithi133.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 400 kB/s | 16 kB 00:00 2023-06-22T17:59:05.228 INFO:teuthology.orchestra.run.smithi101.stdout:(4/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 405 kB/s | 16 kB 00:00 2023-06-22T17:59:05.258 INFO:teuthology.orchestra.run.smithi133.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 1.3 MB/s | 51 kB 00:00 2023-06-22T17:59:05.260 INFO:teuthology.orchestra.run.smithi101.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 1.6 MB/s | 51 kB 00:00 2023-06-22T17:59:05.316 INFO:teuthology.orchestra.run.smithi133.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 57 MB/s | 7.0 MB 00:00 2023-06-22T17:59:05.320 INFO:teuthology.orchestra.run.smithi101.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 59 MB/s | 7.0 MB 00:00 2023-06-22T17:59:05.344 INFO:teuthology.orchestra.run.smithi133.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 55 MB/s | 8.2 MB 00:00 2023-06-22T17:59:05.344 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:59:05.345 INFO:teuthology.orchestra.run.smithi133.stdout:Total 70 MB/s | 16 MB 00:00 2023-06-22T17:59:05.348 INFO:teuthology.orchestra.run.smithi101.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 56 MB/s | 8.2 MB 00:00 2023-06-22T17:59:05.349 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:59:05.349 INFO:teuthology.orchestra.run.smithi101.stdout:Total 72 MB/s | 16 MB 00:00 2023-06-22T17:59:05.350 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-06-22T17:59:05.355 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-06-22T17:59:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-06-22T17:59:05.367 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-06-22T17:59:05.371 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-06-22T17:59:05.372 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-06-22T17:59:06.098 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-06-22T17:59:06.102 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-06-22T17:59:06.206 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-06-22T17:59:06.210 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-06-22T17:59:06.400 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-06-22T17:59:06.457 INFO:teuthology.orchestra.run.smithi039.stdout:========================================================================================================= 2023-06-22T17:59:06.457 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout:========================================================================================================= 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout:Installing group/module packages: 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout: buildah x86_64 1.21.4-2.module+el8.4.0+12050+ef972f71 rhel-8-for-x86_64-appstream-rpms 8.2 M 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout: cockpit-podman noarch 32-2.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 410 k 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout: crun x86_64 0.20.1-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 192 k 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout: skopeo x86_64 1:1.3.1-5.module+el8.4.0+11990+22932769 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout: toolbox noarch 0.0.8-1.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 16 k 2023-06-22T17:59:06.458 INFO:teuthology.orchestra.run.smithi039.stdout: udica noarch 0.2.4-2.module+el8.4.0+11822+6cc1e7d7 rhel-8-for-x86_64-appstream-rpms 51 k 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout:Installing module profiles: 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools/common 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling module streams: 2023-06-22T17:59:06.459 INFO:teuthology.orchestra.run.smithi039.stdout: container-tools rhel8 2023-06-22T17:59:06.460 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:06.460 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2023-06-22T17:59:06.460 INFO:teuthology.orchestra.run.smithi039.stdout:========================================================================================================= 2023-06-22T17:59:06.460 INFO:teuthology.orchestra.run.smithi039.stdout:Install 7 Packages 2023-06-22T17:59:06.460 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:06.461 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 16 M 2023-06-22T17:59:06.461 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 57 M 2023-06-22T17:59:06.461 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2023-06-22T17:59:06.494 INFO:teuthology.orchestra.run.smithi039.stdout:(1/7): yajl-2.1.0-10.el8.x86_64.rpm 1.2 MB/s | 41 kB 00:00 2023-06-22T17:59:06.506 INFO:teuthology.orchestra.run.smithi039.stdout:(2/7): crun-0.20.1-1.module+el8.4.0+11822+6cc1e 4.0 MB/s | 192 kB 00:00 2023-06-22T17:59:06.513 INFO:teuthology.orchestra.run.smithi039.stdout:(3/7): toolbox-0.0.8-1.module+el8.4.0+11822+6cc 837 kB/s | 16 kB 00:00 2023-06-22T17:59:06.516 INFO:teuthology.orchestra.run.smithi039.stdout:(4/7): cockpit-podman-32-2.module+el8.4.0+11990 7.1 MB/s | 410 kB 00:00 2023-06-22T17:59:06.536 INFO:teuthology.orchestra.run.smithi039.stdout:(5/7): udica-0.2.4-2.module+el8.4.0+11822+6cc1e 2.6 MB/s | 51 kB 00:00 2023-06-22T17:59:06.643 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-06-22T17:59:06.672 INFO:teuthology.orchestra.run.smithi039.stdout:(6/7): skopeo-1.3.1-5.module+el8.4.0+11990+2293 43 MB/s | 7.0 MB 00:00 2023-06-22T17:59:06.711 INFO:teuthology.orchestra.run.smithi039.stdout:(7/7): buildah-1.21.4-2.module+el8.4.0+12050+ef 41 MB/s | 8.2 MB 00:00 2023-06-22T17:59:06.712 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:59:06.712 INFO:teuthology.orchestra.run.smithi039.stdout:Total 63 MB/s | 16 MB 00:00 2023-06-22T17:59:06.722 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2023-06-22T17:59:06.744 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2023-06-22T17:59:06.744 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2023-06-22T17:59:06.798 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-06-22T17:59:06.859 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-22T17:59:07.051 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-22T17:59:07.059 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-22T17:59:07.243 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-22T17:59:07.293 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2023-06-22T17:59:07.298 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2023-06-22T17:59:07.731 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2023-06-22T17:59:07.944 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-22T17:59:08.003 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-06-22T17:59:08.140 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-22T17:59:08.197 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-06-22T17:59:08.790 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-06-22T17:59:09.009 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-22T17:59:09.026 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-06-22T17:59:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 3/7 2023-06-22T17:59:09.204 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-06-22T17:59:09.269 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-22T17:59:09.364 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-22T17:59:09.465 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-06-22T17:59:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-22T17:59:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-22T17:59:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-22T17:59:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-06-22T17:59:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-06-22T17:59:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-22T17:59:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-06-22T17:59:09.689 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-22T17:59:09.948 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-22T17:59:09.949 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-22T17:59:09.949 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-22T17:59:09.949 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-06-22T17:59:09.949 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-06-22T17:59:09.949 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-22T17:59:09.950 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-06-22T17:59:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 4/7 2023-06-22T17:59:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-22T17:59:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 6/7 2023-06-22T17:59:10.665 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-22T17:59:10.851 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-06-22T17:59:10.852 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:10:56 ago on Thu 22 Jun 2023 05:48:14 PM UTC. 2023-06-22T17:59:11.021 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 7/7 2023-06-22T17:59:11.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 1/7 2023-06-22T17:59:11.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2/7 2023-06-22T17:59:11.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cockpit-podman-32-2.module+el8.4.0+11990+22932769.no 3/7 2023-06-22T17:59:11.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 4/7 2023-06-22T17:59:11.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_6 5/7 2023-06-22T17:59:11.022 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_6 6/7 2023-06-22T17:59:11.142 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-06-22T17:59:11.142 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:11:00 ago on Thu 22 Jun 2023 05:48:10 PM UTC. 2023-06-22T17:59:12.347 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 7/7 2023-06-22T17:59:12.347 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:02 ago on Thu 22 Jun 2023 05:57:09 PM UTC. 2023-06-22T17:59:13.212 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-06-22T17:59:13.349 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:13.349 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-22T17:59:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-22T17:59:13.351 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:13.351 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-06-22T17:59:13.502 DEBUG:teuthology.parallel:result is None 2023-06-22T17:59:13.615 INFO:teuthology.orchestra.run.smithi101.stdout:Installed products updated. 2023-06-22T17:59:13.745 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:13.746 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-06-22T17:59:13.746 INFO:teuthology.orchestra.run.smithi101.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-06-22T17:59:13.746 INFO:teuthology.orchestra.run.smithi101.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-06-22T17:59:13.746 INFO:teuthology.orchestra.run.smithi101.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-06-22T17:59:13.746 INFO:teuthology.orchestra.run.smithi101.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-06-22T17:59:13.747 INFO:teuthology.orchestra.run.smithi101.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-22T17:59:13.747 INFO:teuthology.orchestra.run.smithi101.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-22T17:59:13.747 INFO:teuthology.orchestra.run.smithi101.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-22T17:59:13.747 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:13.747 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-06-22T17:59:13.892 DEBUG:teuthology.parallel:result is None 2023-06-22T17:59:14.497 INFO:teuthology.orchestra.run.smithi039.stdout:Installed products updated. 2023-06-22T17:59:14.624 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:14.625 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2023-06-22T17:59:14.625 INFO:teuthology.orchestra.run.smithi039.stdout: buildah-1.21.4-2.module+el8.4.0+12050+ef972f71.x86_64 2023-06-22T17:59:14.625 INFO:teuthology.orchestra.run.smithi039.stdout: cockpit-podman-32-2.module+el8.4.0+11990+22932769.noarch 2023-06-22T17:59:14.625 INFO:teuthology.orchestra.run.smithi039.stdout: crun-0.20.1-1.module+el8.4.0+11822+6cc1e7d7.x86_64 2023-06-22T17:59:14.625 INFO:teuthology.orchestra.run.smithi039.stdout: skopeo-1:1.3.1-5.module+el8.4.0+11990+22932769.x86_64 2023-06-22T17:59:14.625 INFO:teuthology.orchestra.run.smithi039.stdout: toolbox-0.0.8-1.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-22T17:59:14.626 INFO:teuthology.orchestra.run.smithi039.stdout: udica-0.2.4-2.module+el8.4.0+11822+6cc1e7d7.noarch 2023-06-22T17:59:14.626 INFO:teuthology.orchestra.run.smithi039.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-22T17:59:14.626 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:14.626 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-06-22T17:59:14.808 DEBUG:teuthology.parallel:result is None 2023-06-22T17:59:14.809 INFO:teuthology.run_tasks:Running task install... 2023-06-22T17:59:14.817 DEBUG:teuthology.task.install:project ceph 2023-06-22T17:59:14.818 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}} 2023-06-22T17:59:14.818 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'} 2023-06-22T17:59:14.818 INFO:teuthology.task.install:Using flavor: default 2023-06-22T17:59:14.824 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-22T17:59:14.825 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-22T17:59:14.825 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-22T17:59:14.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:59:14.828 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-22T17:59:14.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:59:14.830 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-22T17:59:14.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T17:59:14.970 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-22T17:59:14.970 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-22T17:59:14.983 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-22T17:59:14.984 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-22T17:59:15.040 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-22T17:59:15.041 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-22T17:59:15.145 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-22T17:59:15.146 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T17:59:15.146 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-22T17:59:15.151 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-22T17:59:15.151 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T17:59:15.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-22T17:59:15.169 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-22T17:59:15.169 DEBUG:teuthology.orchestra.run.smithi101:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-22T17:59:15.175 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-22T17:59:15.175 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-22T17:59:15.210 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-22T17:59:15.210 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T17:59:15.210 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-22T17:59:15.236 DEBUG:teuthology.orchestra.run.smithi101:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-22T17:59:15.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-22T17:59:15.273 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-22T17:59:15.274 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-22T17:59:15.311 DEBUG:teuthology.orchestra.run.smithi101:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-22T17:59:15.313 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-22T17:59:15.373 INFO:teuthology.orchestra.run.smithi101.stdout:check_obsoletes = 1 2023-06-22T17:59:15.374 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum clean all 2023-06-22T17:59:15.377 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-22T17:59:15.378 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2023-06-22T17:59:15.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2023-06-22T17:59:15.473 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-22T17:59:15.540 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2023-06-22T17:59:15.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2023-06-22T17:59:15.778 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:15.809 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:16.044 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:16.110 INFO:teuthology.orchestra.run.smithi133.stdout:44 files removed 2023-06-22T17:59:16.118 INFO:teuthology.orchestra.run.smithi101.stdout:44 files removed 2023-06-22T17:59:16.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-22T17:59:16.173 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-22T17:59:16.423 INFO:teuthology.orchestra.run.smithi039.stdout:44 files removed 2023-06-22T17:59:16.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-22T17:59:16.528 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:16.529 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:16.995 INFO:teuthology.orchestra.run.smithi039.stdout:Updating Subscription Management repositories. 2023-06-22T17:59:17.250 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 355 kB/s | 78 kB 00:00 2023-06-22T17:59:17.378 INFO:teuthology.orchestra.run.smithi101.stdout:ceph packages for x86_64 250 kB/s | 78 kB 00:00 2023-06-22T17:59:17.534 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-06-22T17:59:17.670 INFO:teuthology.orchestra.run.smithi101.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-06-22T17:59:17.871 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-06-22T17:59:17.891 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 231 kB/s | 78 kB 00:00 2023-06-22T17:59:17.950 INFO:teuthology.orchestra.run.smithi101.stdout:ceph source packages 5.9 kB/s | 1.4 kB 00:00 2023-06-22T17:59:18.184 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-06-22T17:59:18.465 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 5.8 kB/s | 1.4 kB 00:00 2023-06-22T17:59:19.789 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 1.9 kB/s | 3.5 kB 00:01 2023-06-22T17:59:20.317 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-22T17:59:20.625 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-06-22T17:59:20.921 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-22T17:59:22.611 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 16 MB 00:02 2023-06-22T17:59:23.074 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 16 MB 00:02 2023-06-22T17:59:24.294 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 1.7 MB/s | 24 kB 00:00 2023-06-22T17:59:24.704 INFO:teuthology.orchestra.run.smithi101.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 120 MB/s | 38 MB 00:00 2023-06-22T17:59:26.176 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-06-22T17:59:26.551 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 129 MB/s | 38 MB 00:00 2023-06-22T17:59:26.750 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-22T17:59:27.264 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 77 MB/s | 33 MB 00:00 2023-06-22T17:59:30.427 INFO:teuthology.orchestra.run.smithi101.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 126 MB/s | 33 MB 00:00 2023-06-22T17:59:32.005 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 123 MB/s | 33 MB 00:00 2023-06-22T17:59:33.221 INFO:teuthology.orchestra.run.smithi039.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 77 MB/s | 38 MB 00:00 2023-06-22T17:59:38.357 INFO:teuthology.orchestra.run.smithi101.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:59:38.358 INFO:teuthology.orchestra.run.smithi101.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:59:38.369 INFO:teuthology.orchestra.run.smithi101.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-22T17:59:38.369 INFO:teuthology.orchestra.run.smithi101.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-22T17:59:38.370 INFO:teuthology.orchestra.run.smithi101.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-22T17:59:38.371 INFO:teuthology.orchestra.run.smithi101.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-22T17:59:38.373 INFO:teuthology.orchestra.run.smithi101.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-22T17:59:38.375 INFO:teuthology.orchestra.run.smithi101.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-22T17:59:38.376 INFO:teuthology.orchestra.run.smithi101.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-22T17:59:38.856 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-06-22T17:59:38.868 INFO:teuthology.orchestra.run.smithi101.stdout:====================================================================================================================== 2023-06-22T17:59:38.868 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-06-22T17:59:38.868 INFO:teuthology.orchestra.run.smithi101.stdout:====================================================================================================================== 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-22T17:59:38.869 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-22T17:59:38.870 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-22T17:59:38.871 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-22T17:59:38.872 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrading: 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:59:38.873 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-22T17:59:38.874 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-22T17:59:38.875 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-22T17:59:38.876 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-22T17:59:38.877 INFO:teuthology.orchestra.run.smithi101.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-22T17:59:38.878 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-22T17:59:38.879 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-22T17:59:38.880 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-22T17:59:38.881 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:59:38.882 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-22T17:59:38.883 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-22T17:59:38.884 INFO:teuthology.orchestra.run.smithi101.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-22T17:59:38.885 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:38.885 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-06-22T17:59:38.885 INFO:teuthology.orchestra.run.smithi101.stdout:====================================================================================================================== 2023-06-22T17:59:38.885 INFO:teuthology.orchestra.run.smithi101.stdout:Install 118 Packages 2023-06-22T17:59:38.885 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrade 2 Packages 2023-06-22T17:59:38.885 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T17:59:38.886 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 201 M 2023-06-22T17:59:38.886 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-06-22T17:59:39.379 INFO:teuthology.orchestra.run.smithi101.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 39 kB/s | 6.4 kB 00:00 2023-06-22T17:59:39.666 INFO:teuthology.orchestra.run.smithi101.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 3.0 MB/s | 873 kB 00:00 2023-06-22T17:59:39.710 INFO:teuthology.orchestra.run.smithi101.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 3.6 MB/s | 162 kB 00:00 2023-06-22T17:59:39.893 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:59:39.894 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:59:39.904 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-22T17:59:39.905 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-22T17:59:39.906 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-22T17:59:39.907 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-22T17:59:39.909 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-22T17:59:39.911 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-22T17:59:39.912 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-22T17:59:39.950 INFO:teuthology.orchestra.run.smithi101.stdout:(4/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 7.0 MB/s | 5.2 MB 00:00 2023-06-22T17:59:39.967 INFO:teuthology.orchestra.run.smithi101.stdout:(5/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 8.9 MB/s | 2.3 MB 00:00 2023-06-22T17:59:40.102 INFO:teuthology.orchestra.run.smithi101.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 10 MB/s | 1.6 MB 00:00 2023-06-22T17:59:40.401 INFO:teuthology.orchestra.run.smithi101.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 9.1 MB/s | 3.9 MB 00:00 2023-06-22T17:59:40.407 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-06-22T17:59:40.419 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================================================== 2023-06-22T17:59:40.419 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-06-22T17:59:40.419 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================================================== 2023-06-22T17:59:40.419 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2023-06-22T17:59:40.419 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-22T17:59:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-22T17:59:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-22T17:59:40.422 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-22T17:59:40.423 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:59:40.424 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-22T17:59:40.425 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-22T17:59:40.426 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-22T17:59:40.427 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-22T17:59:40.428 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-22T17:59:40.429 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-22T17:59:40.430 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-22T17:59:40.431 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-22T17:59:40.432 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-22T17:59:40.433 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-22T17:59:40.434 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-22T17:59:40.435 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-22T17:59:40.436 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-22T17:59:40.437 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:40.437 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-06-22T17:59:40.437 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================================================== 2023-06-22T17:59:40.437 INFO:teuthology.orchestra.run.smithi133.stdout:Install 118 Packages 2023-06-22T17:59:40.437 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 2 Packages 2023-06-22T17:59:40.437 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T17:59:40.438 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 201 M 2023-06-22T17:59:40.438 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-06-22T17:59:41.156 INFO:teuthology.orchestra.run.smithi133.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 19 kB/s | 6.4 kB 00:00 2023-06-22T17:59:41.891 INFO:teuthology.orchestra.run.smithi101.stdout:(8/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86_ 9.3 MB/s | 17 MB 00:01 2023-06-22T17:59:41.894 INFO:teuthology.orchestra.run.smithi133.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 1.2 MB/s | 873 kB 00:00 2023-06-22T17:59:41.965 INFO:teuthology.orchestra.run.smithi101.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 331 kB/s | 24 kB 00:00 2023-06-22T17:59:42.061 INFO:teuthology.orchestra.run.smithi133.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 975 kB/s | 162 kB 00:00 2023-06-22T17:59:42.471 INFO:teuthology.orchestra.run.smithi101.stdout:(10/120): ceph-common-17.2.6-719.g42ddc46e.el8. 7.2 MB/s | 23 MB 00:03 2023-06-22T17:59:42.589 INFO:teuthology.orchestra.run.smithi101.stdout:(11/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 7.7 MB/s | 17 MB 00:02 2023-06-22T17:59:42.592 INFO:teuthology.orchestra.run.smithi101.stdout:(12/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 2.1 MB/s | 255 kB 00:00 2023-06-22T17:59:42.604 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:59:42.605 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-22T17:59:42.616 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-22T17:59:42.617 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-22T17:59:42.618 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-22T17:59:42.619 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-22T17:59:42.620 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-22T17:59:42.623 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-22T17:59:42.624 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-22T17:59:42.632 INFO:teuthology.orchestra.run.smithi101.stdout:(13/120): libcephfs-devel-17.2.6-719.g42ddc46e. 580 kB/s | 25 kB 00:00 2023-06-22T17:59:42.714 INFO:teuthology.orchestra.run.smithi101.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 2.1 MB/s | 174 kB 00:00 2023-06-22T17:59:42.717 INFO:teuthology.orchestra.run.smithi101.stdout:(15/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 6.2 MB/s | 794 kB 00:00 2023-06-22T17:59:42.764 INFO:teuthology.orchestra.run.smithi133.stdout:(4/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 2.7 MB/s | 5.2 MB 00:01 2023-06-22T17:59:42.766 INFO:teuthology.orchestra.run.smithi101.stdout:(16/120): librados-devel-17.2.6-719.g42ddc46e.e 2.4 MB/s | 130 kB 00:00 2023-06-22T17:59:42.787 INFO:teuthology.orchestra.run.smithi101.stdout:(17/120): libradosstriper1-17.2.6-719.g42ddc46e 7.1 MB/s | 506 kB 00:00 2023-06-22T17:59:42.853 INFO:teuthology.orchestra.run.smithi101.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 676 kB/s | 45 kB 00:00 2023-06-22T17:59:42.926 INFO:teuthology.orchestra.run.smithi101.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.6 MB/s | 115 kB 00:00 2023-06-22T17:59:43.003 INFO:teuthology.orchestra.run.smithi101.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 2.4 MB/s | 190 kB 00:00 2023-06-22T17:59:43.089 INFO:teuthology.orchestra.run.smithi101.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 4.2 MB/s | 371 kB 00:00 2023-06-22T17:59:43.146 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2023-06-22T17:59:43.159 INFO:teuthology.orchestra.run.smithi039.stdout:====================================================================================================================== 2023-06-22T17:59:43.160 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2023-06-22T17:59:43.160 INFO:teuthology.orchestra.run.smithi039.stdout:====================================================================================================================== 2023-06-22T17:59:43.160 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2023-06-22T17:59:43.161 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-22T17:59:43.161 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-22T17:59:43.161 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-22T17:59:43.161 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-22T17:59:43.161 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-22T17:59:43.162 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-22T17:59:43.162 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-22T17:59:43.162 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-22T17:59:43.162 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-22T17:59:43.163 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-22T17:59:43.163 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:59:43.163 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-22T17:59:43.163 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-22T17:59:43.164 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-22T17:59:43.164 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-22T17:59:43.164 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-22T17:59:43.164 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-22T17:59:43.164 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-22T17:59:43.165 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-22T17:59:43.165 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-22T17:59:43.165 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-22T17:59:43.165 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-22T17:59:43.166 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-22T17:59:43.166 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-22T17:59:43.166 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-22T17:59:43.166 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-22T17:59:43.166 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-22T17:59:43.167 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-22T17:59:43.167 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-22T17:59:43.167 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-22T17:59:43.167 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:59:43.167 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-22T17:59:43.168 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-22T17:59:43.168 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2023-06-22T17:59:43.168 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-22T17:59:43.168 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-22T17:59:43.169 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2023-06-22T17:59:43.169 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-22T17:59:43.169 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-22T17:59:43.169 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-22T17:59:43.169 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-22T17:59:43.170 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-22T17:59:43.170 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-22T17:59:43.170 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-22T17:59:43.170 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-22T17:59:43.171 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-22T17:59:43.171 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-22T17:59:43.171 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-22T17:59:43.171 INFO:teuthology.orchestra.run.smithi039.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-22T17:59:43.171 INFO:teuthology.orchestra.run.smithi039.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-22T17:59:43.172 INFO:teuthology.orchestra.run.smithi039.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-22T17:59:43.172 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-22T17:59:43.172 INFO:teuthology.orchestra.run.smithi039.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-22T17:59:43.172 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-22T17:59:43.172 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-22T17:59:43.172 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-22T17:59:43.173 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-22T17:59:43.173 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-22T17:59:43.173 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-22T17:59:43.173 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-22T17:59:43.173 INFO:teuthology.orchestra.run.smithi039.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-22T17:59:43.174 INFO:teuthology.orchestra.run.smithi039.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-22T17:59:43.174 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-22T17:59:43.174 INFO:teuthology.orchestra.run.smithi039.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-22T17:59:43.174 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-22T17:59:43.174 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-22T17:59:43.174 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-22T17:59:43.175 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-22T17:59:43.176 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:59:43.177 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-22T17:59:43.178 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-22T17:59:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-22T17:59:43.180 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-22T17:59:43.181 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2023-06-22T17:59:43.182 INFO:teuthology.orchestra.run.smithi039.stdout:====================================================================================================================== 2023-06-22T17:59:43.183 INFO:teuthology.orchestra.run.smithi039.stdout:Install 118 Packages 2023-06-22T17:59:43.183 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 2 Packages 2023-06-22T17:59:43.183 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T17:59:43.183 INFO:teuthology.orchestra.run.smithi101.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 4.1 MB/s | 357 kB 00:00 2023-06-22T17:59:43.184 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 201 M 2023-06-22T17:59:43.184 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2023-06-22T17:59:43.252 INFO:teuthology.orchestra.run.smithi101.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 1.3 MB/s | 107 kB 00:00 2023-06-22T17:59:43.300 INFO:teuthology.orchestra.run.smithi133.stdout:(5/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 2.9 MB/s | 1.6 MB 00:00 2023-06-22T17:59:43.324 INFO:teuthology.orchestra.run.smithi101.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.2 MB/s | 91 kB 00:00 2023-06-22T17:59:43.544 INFO:teuthology.orchestra.run.smithi133.stdout:(6/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 1.5 MB/s | 2.3 MB 00:01 2023-06-22T17:59:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 20 kB/s | 6.4 kB 00:00 2023-06-22T17:59:43.914 INFO:teuthology.orchestra.run.smithi101.stdout:(25/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 5.9 MB/s | 3.5 MB 00:00 2023-06-22T17:59:43.983 INFO:teuthology.orchestra.run.smithi101.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 2.4 MB/s | 173 kB 00:00 2023-06-22T17:59:44.039 INFO:teuthology.orchestra.run.smithi101.stdout:(27/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 4.4 MB/s | 5.5 MB 00:01 2023-06-22T17:59:44.047 INFO:teuthology.orchestra.run.smithi101.stdout:(28/120): ceph-grafana-dashboards-17.2.6-719.g4 360 kB/s | 23 kB 00:00 2023-06-22T17:59:44.104 INFO:teuthology.orchestra.run.smithi101.stdout:(29/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 1.8 MB/s | 121 kB 00:00 2023-06-22T17:59:44.421 INFO:teuthology.orchestra.run.smithi101.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 4.5 MB/s | 1.7 MB 00:00 2023-06-22T17:59:44.479 INFO:teuthology.orchestra.run.smithi039.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 1.3 MB/s | 873 kB 00:00 2023-06-22T17:59:44.501 INFO:teuthology.orchestra.run.smithi101.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 2.9 MB/s | 238 kB 00:00 2023-06-22T17:59:44.558 INFO:teuthology.orchestra.run.smithi039.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 2.0 MB/s | 162 kB 00:00 2023-06-22T17:59:44.573 INFO:teuthology.orchestra.run.smithi101.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 656 kB/s | 47 kB 00:00 2023-06-22T17:59:44.647 INFO:teuthology.orchestra.run.smithi101.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 197 kB/s | 14 kB 00:00 2023-06-22T17:59:44.732 INFO:teuthology.orchestra.run.smithi101.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 3.0 MB/s | 264 kB 00:00 2023-06-22T17:59:44.810 INFO:teuthology.orchestra.run.smithi101.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.0 MB/s | 80 kB 00:00 2023-06-22T17:59:44.833 INFO:teuthology.orchestra.run.smithi133.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 2.6 MB/s | 3.9 MB 00:01 2023-06-22T17:59:44.881 INFO:teuthology.orchestra.run.smithi101.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 252 kB/s | 18 kB 00:00 2023-06-22T17:59:45.115 INFO:teuthology.orchestra.run.smithi101.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-06-22T17:59:45.156 INFO:teuthology.orchestra.run.smithi101.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-06-22T17:59:45.194 INFO:teuthology.orchestra.run.smithi101.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.9 MB/s | 75 kB 00:00 2023-06-22T17:59:45.271 INFO:teuthology.orchestra.run.smithi101.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 6.1 MB/s | 476 kB 00:00 2023-06-22T17:59:45.286 INFO:teuthology.orchestra.run.smithi133.stdout:(8/120): ceph-common-17.2.6-719.g42ddc46e.el8.x 5.2 MB/s | 23 MB 00:04 2023-06-22T17:59:45.308 INFO:teuthology.orchestra.run.smithi101.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.1 MB/s | 44 kB 00:00 2023-06-22T17:59:45.346 INFO:teuthology.orchestra.run.smithi133.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 405 kB/s | 24 kB 00:00 2023-06-22T17:59:45.348 INFO:teuthology.orchestra.run.smithi039.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 2.9 MB/s | 2.3 MB 00:00 2023-06-22T17:59:45.348 INFO:teuthology.orchestra.run.smithi101.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 4.5 MB/s | 185 kB 00:00 2023-06-22T17:59:45.386 INFO:teuthology.orchestra.run.smithi101.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 883 kB/s | 33 kB 00:00 2023-06-22T17:59:45.424 INFO:teuthology.orchestra.run.smithi101.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 437 kB/s | 16 kB 00:00 2023-06-22T17:59:45.464 INFO:teuthology.orchestra.run.smithi101.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.2 MB/s | 173 kB 00:00 2023-06-22T17:59:45.508 INFO:teuthology.orchestra.run.smithi101.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.7 MB/s | 384 kB 00:00 2023-06-22T17:59:45.547 INFO:teuthology.orchestra.run.smithi101.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 2.1 MB/s | 81 kB 00:00 2023-06-22T17:59:45.573 INFO:teuthology.orchestra.run.smithi039.stdout:(5/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 2.5 MB/s | 5.2 MB 00:02 2023-06-22T17:59:45.584 INFO:teuthology.orchestra.run.smithi101.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 300 kB/s | 11 kB 00:00 2023-06-22T17:59:45.621 INFO:teuthology.orchestra.run.smithi101.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 474 kB/s | 18 kB 00:00 2023-06-22T17:59:45.703 INFO:teuthology.orchestra.run.smithi101.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 14 MB/s | 1.2 MB 00:00 2023-06-22T17:59:45.741 INFO:teuthology.orchestra.run.smithi101.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.3 MB/s | 49 kB 00:00 2023-06-22T17:59:45.779 INFO:teuthology.orchestra.run.smithi101.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.5 MB/s | 59 kB 00:00 2023-06-22T17:59:45.817 INFO:teuthology.orchestra.run.smithi101.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.6 MB/s | 60 kB 00:00 2023-06-22T17:59:45.859 INFO:teuthology.orchestra.run.smithi101.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2023-06-22T17:59:45.896 INFO:teuthology.orchestra.run.smithi101.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 419 kB/s | 16 kB 00:00 2023-06-22T17:59:45.908 INFO:teuthology.orchestra.run.smithi039.stdout:(6/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 2.8 MB/s | 1.6 MB 00:00 2023-06-22T17:59:45.934 INFO:teuthology.orchestra.run.smithi101.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 892 kB/s | 34 kB 00:00 2023-06-22T17:59:45.974 INFO:teuthology.orchestra.run.smithi101.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 4.8 MB/s | 196 kB 00:00 2023-06-22T17:59:46.012 INFO:teuthology.orchestra.run.smithi101.stdout:(58/120): python3-rsa-4.8-1.el8.noarch.rpm 1.6 MB/s | 61 kB 00:00 2023-06-22T17:59:46.052 INFO:teuthology.orchestra.run.smithi101.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 475 kB/s | 19 kB 00:00 2023-06-22T17:59:46.090 INFO:teuthology.orchestra.run.smithi101.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 639 kB/s | 24 kB 00:00 2023-06-22T17:59:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 786 kB/s | 29 kB 00:00 2023-06-22T17:59:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 674 kB/s | 27 kB 00:00 2023-06-22T17:59:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 5.4 MB/s | 255 kB 00:00 2023-06-22T17:59:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 5.5 MB/s | 251 kB 00:00 2023-06-22T17:59:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 1.6 MB/s | 61 kB 00:00 2023-06-22T17:59:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 2.2 MB/s | 85 kB 00:00 2023-06-22T17:59:46.373 INFO:teuthology.orchestra.run.smithi101.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 616 kB/s | 23 kB 00:00 2023-06-22T17:59:46.440 INFO:teuthology.orchestra.run.smithi101.stdout:(68/120): thrift-0.13.0-2.el8.x86_64.rpm 26 MB/s | 1.7 MB 00:00 2023-06-22T17:59:46.443 INFO:teuthology.orchestra.run.smithi101.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 21 MB/s | 70 kB 00:00 2023-06-22T17:59:46.475 INFO:teuthology.orchestra.run.smithi101.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 594 kB/s | 19 kB 00:00 2023-06-22T17:59:46.489 INFO:teuthology.orchestra.run.smithi101.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.0 MB/s | 43 kB 00:00 2023-06-22T17:59:46.504 INFO:teuthology.orchestra.run.smithi101.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-06-22T17:59:46.520 INFO:teuthology.orchestra.run.smithi101.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-06-22T17:59:46.534 INFO:teuthology.orchestra.run.smithi101.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.4 MB/s | 48 kB 00:00 2023-06-22T17:59:46.550 INFO:teuthology.orchestra.run.smithi101.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-06-22T17:59:46.562 INFO:teuthology.orchestra.run.smithi101.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.4 MB/s | 43 kB 00:00 2023-06-22T17:59:46.597 INFO:teuthology.orchestra.run.smithi101.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 67 MB/s | 2.3 MB 00:00 2023-06-22T17:59:46.622 INFO:teuthology.orchestra.run.smithi101.stdout:(78/120): ceph-mgr-diskprediction-local-17.2.6- 2.9 MB/s | 7.4 MB 00:02 2023-06-22T17:59:46.623 INFO:teuthology.orchestra.run.smithi101.stdout:(79/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 672 kB/s | 18 kB 00:00 2023-06-22T17:59:46.638 INFO:teuthology.orchestra.run.smithi101.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 5.8 MB/s | 83 kB 00:00 2023-06-22T17:59:46.652 INFO:teuthology.orchestra.run.smithi101.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.4 MB/s | 59 kB 00:00 2023-06-22T17:59:46.656 INFO:teuthology.orchestra.run.smithi101.stdout:(82/120): pcre2-devel-10.32-2.el8.x86_64.rpm 18 MB/s | 605 kB 00:00 2023-06-22T17:59:46.665 INFO:teuthology.orchestra.run.smithi101.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-06-22T17:59:46.671 INFO:teuthology.orchestra.run.smithi101.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.6 MB/s | 38 kB 00:00 2023-06-22T17:59:46.679 INFO:teuthology.orchestra.run.smithi101.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-06-22T17:59:46.684 INFO:teuthology.orchestra.run.smithi101.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.5 MB/s | 58 kB 00:00 2023-06-22T17:59:46.692 INFO:teuthology.orchestra.run.smithi101.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.8 MB/s | 47 kB 00:00 2023-06-22T17:59:46.701 INFO:teuthology.orchestra.run.smithi101.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 32 MB/s | 559 kB 00:00 2023-06-22T17:59:46.710 INFO:teuthology.orchestra.run.smithi101.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 35 MB/s | 639 kB 00:00 2023-06-22T17:59:46.714 INFO:teuthology.orchestra.run.smithi101.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.5 MB/s | 70 kB 00:00 2023-06-22T17:59:46.724 INFO:teuthology.orchestra.run.smithi101.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 14 MB/s | 200 kB 00:00 2023-06-22T17:59:46.725 INFO:teuthology.orchestra.run.smithi039.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 3.4 MB/s | 3.9 MB 00:01 2023-06-22T17:59:46.728 INFO:teuthology.orchestra.run.smithi101.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.7 MB/s | 97 kB 00:00 2023-06-22T17:59:46.744 INFO:teuthology.orchestra.run.smithi101.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.4 MB/s | 44 kB 00:00 2023-06-22T17:59:46.754 INFO:teuthology.orchestra.run.smithi101.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.0 MB/s | 27 kB 00:00 2023-06-22T17:59:46.768 INFO:teuthology.orchestra.run.smithi101.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-06-22T17:59:46.783 INFO:teuthology.orchestra.run.smithi101.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 21 MB/s | 320 kB 00:00 2023-06-22T17:59:46.801 INFO:teuthology.orchestra.run.smithi101.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 25 MB/s | 457 kB 00:00 2023-06-22T17:59:46.874 INFO:teuthology.orchestra.run.smithi101.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 66 MB/s | 4.8 MB 00:00 2023-06-22T17:59:46.985 INFO:teuthology.orchestra.run.smithi101.stdout:(99/120): python3-scipy-1.0.0-20.module+el8.1.0 56 MB/s | 15 MB 00:00 2023-06-22T17:59:46.986 INFO:teuthology.orchestra.run.smithi101.stdout:(100/120): python3-markupsafe-0.23-19.el8.x86_6 347 kB/s | 39 kB 00:00 2023-06-22T17:59:47.002 INFO:teuthology.orchestra.run.smithi101.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-06-22T17:59:47.007 INFO:teuthology.orchestra.run.smithi101.stdout:(102/120): bison-3.0.4-10.el8.x86_64.rpm 31 MB/s | 688 kB 00:00 2023-06-22T17:59:47.018 INFO:teuthology.orchestra.run.smithi101.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 14 MB/s | 225 kB 00:00 2023-06-22T17:59:47.022 INFO:teuthology.orchestra.run.smithi101.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.8 MB/s | 126 kB 00:00 2023-06-22T17:59:47.036 INFO:teuthology.orchestra.run.smithi101.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-06-22T17:59:47.055 INFO:teuthology.orchestra.run.smithi101.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 28 MB/s | 538 kB 00:00 2023-06-22T17:59:47.073 INFO:teuthology.orchestra.run.smithi101.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 68 MB/s | 3.7 MB 00:00 2023-06-22T17:59:47.077 INFO:teuthology.orchestra.run.smithi101.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 14 MB/s | 302 kB 00:00 2023-06-22T17:59:47.087 INFO:teuthology.orchestra.run.smithi101.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-06-22T17:59:47.092 INFO:teuthology.orchestra.run.smithi101.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.4 MB/s | 110 kB 00:00 2023-06-22T17:59:47.104 INFO:teuthology.orchestra.run.smithi101.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 21 MB/s | 353 kB 00:00 2023-06-22T17:59:47.105 INFO:teuthology.orchestra.run.smithi101.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.6 MB/s | 20 kB 00:00 2023-06-22T17:59:47.117 INFO:teuthology.orchestra.run.smithi101.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.3 MB/s | 54 kB 00:00 2023-06-22T17:59:47.124 INFO:teuthology.orchestra.run.smithi101.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 35 MB/s | 709 kB 00:00 2023-06-22T17:59:47.187 INFO:teuthology.orchestra.run.smithi101.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 68 MB/s | 4.7 MB 00:00 2023-06-22T17:59:47.205 INFO:teuthology.orchestra.run.smithi101.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 57 MB/s | 4.6 MB 00:00 2023-06-22T17:59:47.212 INFO:teuthology.orchestra.run.smithi101.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 28 MB/s | 713 kB 00:00 2023-06-22T17:59:48.532 INFO:teuthology.orchestra.run.smithi101.stdout:(118/120): librados2-17.2.6-719.g42ddc46e.el8.x 2.6 MB/s | 3.5 MB 00:01 2023-06-22T17:59:48.793 INFO:teuthology.orchestra.run.smithi101.stdout:(119/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 2.3 MB/s | 3.6 MB 00:01 2023-06-22T17:59:50.200 INFO:teuthology.orchestra.run.smithi039.stdout:(8/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86_ 3.9 MB/s | 17 MB 00:04 2023-06-22T17:59:50.261 INFO:teuthology.orchestra.run.smithi039.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 402 kB/s | 24 kB 00:00 2023-06-22T17:59:50.461 INFO:teuthology.orchestra.run.smithi039.stdout:(10/120): ceph-common-17.2.6-719.g42ddc46e.el8. 3.4 MB/s | 23 MB 00:06 2023-06-22T17:59:50.536 INFO:teuthology.orchestra.run.smithi039.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 3.3 MB/s | 255 kB 00:00 2023-06-22T17:59:50.591 INFO:teuthology.orchestra.run.smithi039.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 453 kB/s | 25 kB 00:00 2023-06-22T17:59:50.633 INFO:teuthology.orchestra.run.smithi133.stdout:(10/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 2.9 MB/s | 17 MB 00:05 2023-06-22T17:59:50.701 INFO:teuthology.orchestra.run.smithi039.stdout:(13/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 4.2 MB/s | 17 MB 00:03 2023-06-22T17:59:50.735 INFO:teuthology.orchestra.run.smithi133.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 2.4 MB/s | 255 kB 00:00 2023-06-22T17:59:50.755 INFO:teuthology.orchestra.run.smithi039.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 3.2 MB/s | 174 kB 00:00 2023-06-22T17:59:50.776 INFO:teuthology.orchestra.run.smithi039.stdout:(15/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 4.2 MB/s | 794 kB 00:00 2023-06-22T17:59:50.782 INFO:teuthology.orchestra.run.smithi133.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 539 kB/s | 25 kB 00:00 2023-06-22T17:59:50.803 INFO:teuthology.orchestra.run.smithi039.stdout:(16/120): librados-devel-17.2.6-719.g42ddc46e.e 2.7 MB/s | 130 kB 00:00 2023-06-22T17:59:50.884 INFO:teuthology.orchestra.run.smithi039.stdout:(17/120): libradosstriper1-17.2.6-719.g42ddc46e 4.5 MB/s | 506 kB 00:00 2023-06-22T17:59:50.939 INFO:teuthology.orchestra.run.smithi039.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 813 kB/s | 45 kB 00:00 2023-06-22T17:59:50.997 INFO:teuthology.orchestra.run.smithi039.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.9 MB/s | 115 kB 00:00 2023-06-22T17:59:51.002 INFO:teuthology.orchestra.run.smithi133.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 3.5 MB/s | 794 kB 00:00 2023-06-22T17:59:51.051 INFO:teuthology.orchestra.run.smithi039.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 3.5 MB/s | 190 kB 00:00 2023-06-22T17:59:51.060 INFO:teuthology.orchestra.run.smithi133.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 2.9 MB/s | 174 kB 00:00 2023-06-22T17:59:51.120 INFO:teuthology.orchestra.run.smithi133.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 2.1 MB/s | 130 kB 00:00 2023-06-22T17:59:51.153 INFO:teuthology.orchestra.run.smithi101.stdout:(120/120): ceph-test-17.2.6-719.g42ddc46e.el8.x 5.6 MB/s | 52 MB 00:09 2023-06-22T17:59:51.157 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:59:51.158 INFO:teuthology.orchestra.run.smithi101.stdout:Total 16 MB/s | 201 MB 00:12 2023-06-22T17:59:51.164 INFO:teuthology.orchestra.run.smithi039.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 3.2 MB/s | 371 kB 00:00 2023-06-22T17:59:51.210 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-06-22T17:59:51.271 INFO:teuthology.orchestra.run.smithi039.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 3.3 MB/s | 357 kB 00:00 2023-06-22T17:59:51.308 INFO:teuthology.orchestra.run.smithi039.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 2.8 MB/s | 107 kB 00:00 2023-06-22T17:59:51.325 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-06-22T17:59:51.325 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-06-22T17:59:51.329 INFO:teuthology.orchestra.run.smithi133.stdout:(16/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 2.1 MB/s | 17 MB 00:07 2023-06-22T17:59:51.333 INFO:teuthology.orchestra.run.smithi133.stdout:(17/120): libradosstriper1-17.2.6-719.g42ddc46e 2.3 MB/s | 506 kB 00:00 2023-06-22T17:59:51.345 INFO:teuthology.orchestra.run.smithi039.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 2.4 MB/s | 91 kB 00:00 2023-06-22T17:59:51.380 INFO:teuthology.orchestra.run.smithi133.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 938 kB/s | 45 kB 00:00 2023-06-22T17:59:51.440 INFO:teuthology.orchestra.run.smithi133.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.9 MB/s | 115 kB 00:00 2023-06-22T17:59:51.502 INFO:teuthology.orchestra.run.smithi133.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 3.0 MB/s | 190 kB 00:00 2023-06-22T17:59:51.626 INFO:teuthology.orchestra.run.smithi133.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 2.9 MB/s | 371 kB 00:00 2023-06-22T17:59:51.737 INFO:teuthology.orchestra.run.smithi039.stdout:(25/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 5.9 MB/s | 5.5 MB 00:00 2023-06-22T17:59:51.742 INFO:teuthology.orchestra.run.smithi133.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 3.0 MB/s | 357 kB 00:00 2023-06-22T17:59:51.790 INFO:teuthology.orchestra.run.smithi039.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 3.2 MB/s | 173 kB 00:00 2023-06-22T17:59:51.795 INFO:teuthology.orchestra.run.smithi133.stdout:(23/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 2.0 MB/s | 107 kB 00:00 2023-06-22T17:59:51.840 INFO:teuthology.orchestra.run.smithi039.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 470 kB/s | 23 kB 00:00 2023-06-22T17:59:51.843 INFO:teuthology.orchestra.run.smithi133.stdout:(24/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.9 MB/s | 91 kB 00:00 2023-06-22T17:59:51.891 INFO:teuthology.orchestra.run.smithi039.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 2.3 MB/s | 121 kB 00:00 2023-06-22T17:59:52.044 INFO:teuthology.orchestra.run.smithi039.stdout:(29/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 5.0 MB/s | 3.5 MB 00:00 2023-06-22T17:59:52.218 INFO:teuthology.orchestra.run.smithi039.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 5.2 MB/s | 1.7 MB 00:00 2023-06-22T17:59:52.292 INFO:teuthology.orchestra.run.smithi039.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 3.2 MB/s | 238 kB 00:00 2023-06-22T17:59:52.347 INFO:teuthology.orchestra.run.smithi039.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 857 kB/s | 47 kB 00:00 2023-06-22T17:59:52.396 INFO:teuthology.orchestra.run.smithi039.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 296 kB/s | 14 kB 00:00 2023-06-22T17:59:52.461 INFO:teuthology.orchestra.run.smithi039.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 4.0 MB/s | 264 kB 00:00 2023-06-22T17:59:52.518 INFO:teuthology.orchestra.run.smithi039.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.4 MB/s | 80 kB 00:00 2023-06-22T17:59:52.574 INFO:teuthology.orchestra.run.smithi039.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 324 kB/s | 18 kB 00:00 2023-06-22T17:59:52.674 INFO:teuthology.orchestra.run.smithi133.stdout:(25/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 4.2 MB/s | 3.5 MB 00:00 2023-06-22T17:59:52.726 INFO:teuthology.orchestra.run.smithi133.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 3.3 MB/s | 173 kB 00:00 2023-06-22T17:59:52.773 INFO:teuthology.orchestra.run.smithi133.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 495 kB/s | 23 kB 00:00 2023-06-22T17:59:52.815 INFO:teuthology.orchestra.run.smithi133.stdout:(28/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 3.7 MB/s | 5.5 MB 00:01 2023-06-22T17:59:52.821 INFO:teuthology.orchestra.run.smithi133.stdout:(29/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 2.5 MB/s | 121 kB 00:00 2023-06-22T17:59:52.836 INFO:teuthology.orchestra.run.smithi039.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-06-22T17:59:52.874 INFO:teuthology.orchestra.run.smithi039.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.5 MB/s | 59 kB 00:00 2023-06-22T17:59:52.913 INFO:teuthology.orchestra.run.smithi039.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.9 MB/s | 75 kB 00:00 2023-06-22T17:59:52.917 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-06-22T17:59:52.921 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-06-22T17:59:53.063 INFO:teuthology.orchestra.run.smithi039.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 3.1 MB/s | 476 kB 00:00 2023-06-22T17:59:53.101 INFO:teuthology.orchestra.run.smithi039.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.1 MB/s | 44 kB 00:00 2023-06-22T17:59:53.177 INFO:teuthology.orchestra.run.smithi039.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-06-22T17:59:53.195 INFO:teuthology.orchestra.run.smithi133.stdout:(30/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 4.5 MB/s | 1.7 MB 00:00 2023-06-22T17:59:53.218 INFO:teuthology.orchestra.run.smithi039.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 804 kB/s | 33 kB 00:00 2023-06-22T17:59:53.253 INFO:teuthology.orchestra.run.smithi133.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 4.0 MB/s | 238 kB 00:00 2023-06-22T17:59:53.259 INFO:teuthology.orchestra.run.smithi039.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 403 kB/s | 16 kB 00:00 2023-06-22T17:59:53.300 INFO:teuthology.orchestra.run.smithi133.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 1.0 MB/s | 47 kB 00:00 2023-06-22T17:59:53.335 INFO:teuthology.orchestra.run.smithi039.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-06-22T17:59:53.345 INFO:teuthology.orchestra.run.smithi133.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 325 kB/s | 14 kB 00:00 2023-06-22T17:59:53.402 INFO:teuthology.orchestra.run.smithi133.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 4.5 MB/s | 264 kB 00:00 2023-06-22T17:59:53.441 INFO:teuthology.orchestra.run.smithi039.stdout:(46/120): ceph-mgr-diskprediction-local-17.2.6- 5.3 MB/s | 7.4 MB 00:01 2023-06-22T17:59:53.451 INFO:teuthology.orchestra.run.smithi039.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-06-22T17:59:53.452 INFO:teuthology.orchestra.run.smithi133.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.6 MB/s | 80 kB 00:00 2023-06-22T17:59:53.488 INFO:teuthology.orchestra.run.smithi133.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 488 kB/s | 18 kB 00:00 2023-06-22T17:59:53.489 INFO:teuthology.orchestra.run.smithi039.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 295 kB/s | 11 kB 00:00 2023-06-22T17:59:53.536 INFO:teuthology.orchestra.run.smithi039.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 379 kB/s | 18 kB 00:00 2023-06-22T17:59:53.610 INFO:teuthology.orchestra.run.smithi039.stdout:(50/120): python3-google-auth-1.1.1-10.el8.noar 481 kB/s | 81 kB 00:00 2023-06-22T17:59:53.652 INFO:teuthology.orchestra.run.smithi039.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-06-22T17:59:53.698 INFO:teuthology.orchestra.run.smithi039.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.3 MB/s | 59 kB 00:00 2023-06-22T17:59:53.742 INFO:teuthology.orchestra.run.smithi039.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.3 MB/s | 60 kB 00:00 2023-06-22T17:59:53.837 INFO:teuthology.orchestra.run.smithi039.stdout:(54/120): python3-kubernetes-11.0.0-6.el8.noarc 3.9 MB/s | 1.2 MB 00:00 2023-06-22T17:59:53.869 INFO:teuthology.orchestra.run.smithi039.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-06-22T17:59:53.878 INFO:teuthology.orchestra.run.smithi039.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 382 kB/s | 16 kB 00:00 2023-06-22T17:59:53.908 INFO:teuthology.orchestra.run.smithi039.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 871 kB/s | 34 kB 00:00 2023-06-22T17:59:53.920 INFO:teuthology.orchestra.run.smithi039.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 4.7 MB/s | 196 kB 00:00 2023-06-22T17:59:53.923 INFO:teuthology.orchestra.run.smithi133.stdout:(37/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 6.0 MB/s | 52 MB 00:08 2023-06-22T17:59:53.949 INFO:teuthology.orchestra.run.smithi039.stdout:(59/120): python3-rsa-4.8-1.el8.noarch.rpm 1.5 MB/s | 61 kB 00:00 2023-06-22T17:59:53.961 INFO:teuthology.orchestra.run.smithi039.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 459 kB/s | 19 kB 00:00 2023-06-22T17:59:53.993 INFO:teuthology.orchestra.run.smithi039.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 546 kB/s | 24 kB 00:00 2023-06-22T17:59:54.007 INFO:teuthology.orchestra.run.smithi039.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 643 kB/s | 29 kB 00:00 2023-06-22T17:59:54.036 INFO:teuthology.orchestra.run.smithi039.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 643 kB/s | 27 kB 00:00 2023-06-22T17:59:54.099 INFO:teuthology.orchestra.run.smithi039.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-06-22T17:59:54.129 INFO:teuthology.orchestra.run.smithi039.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-06-22T17:59:54.147 INFO:teuthology.orchestra.run.smithi039.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-06-22T17:59:54.178 INFO:teuthology.orchestra.run.smithi039.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-06-22T17:59:54.198 INFO:teuthology.orchestra.run.smithi039.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 452 kB/s | 23 kB 00:00 2023-06-22T17:59:54.202 INFO:teuthology.orchestra.run.smithi039.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-06-22T17:59:54.295 INFO:teuthology.orchestra.run.smithi133.stdout:(38/120): ceph-mgr-diskprediction-local-17.2.6- 5.0 MB/s | 7.4 MB 00:01 2023-06-22T17:59:54.398 INFO:teuthology.orchestra.run.smithi133.stdout:(39/120): liboath-2.6.2-3.el8.x86_64.rpm 125 kB/s | 59 kB 00:00 2023-06-22T17:59:54.489 INFO:teuthology.orchestra.run.smithi039.stdout:(70/120): python3-scipy-1.0.0-20.module+el8.1.0 51 MB/s | 15 MB 00:00 2023-06-22T17:59:54.506 INFO:teuthology.orchestra.run.smithi039.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 2.6 MB/s | 44 kB 00:00 2023-06-22T17:59:54.520 INFO:teuthology.orchestra.run.smithi039.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-06-22T17:59:54.536 INFO:teuthology.orchestra.run.smithi039.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 9.9 MB/s | 161 kB 00:00 2023-06-22T17:59:54.555 INFO:teuthology.orchestra.run.smithi039.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-06-22T17:59:54.575 INFO:teuthology.orchestra.run.smithi133.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 282 kB/s | 306 kB 00:01 2023-06-22T17:59:54.577 INFO:teuthology.orchestra.run.smithi039.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 20 MB/s | 457 kB 00:00 2023-06-22T17:59:54.673 INFO:teuthology.orchestra.run.smithi039.stdout:(76/120): python3-babel-2.5.1-5.el8.noarch.rpm 50 MB/s | 4.8 MB 00:00 2023-06-22T17:59:54.675 INFO:teuthology.orchestra.run.smithi133.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 437 kB/s | 44 kB 00:00 2023-06-22T17:59:54.687 INFO:teuthology.orchestra.run.smithi039.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.8 MB/s | 39 kB 00:00 2023-06-22T17:59:54.711 INFO:teuthology.orchestra.run.smithi039.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 28 MB/s | 688 kB 00:00 2023-06-22T17:59:54.728 INFO:teuthology.orchestra.run.smithi039.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-06-22T17:59:54.744 INFO:teuthology.orchestra.run.smithi039.stdout:(80/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 14 MB/s | 225 kB 00:00 2023-06-22T17:59:54.758 INFO:teuthology.orchestra.run.smithi039.stdout:(81/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 9.0 MB/s | 126 kB 00:00 2023-06-22T17:59:54.817 INFO:teuthology.orchestra.run.smithi133.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-06-22T17:59:54.819 INFO:teuthology.orchestra.run.smithi133.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-06-22T17:59:54.832 INFO:teuthology.orchestra.run.smithi133.stdout:(44/120): libunwind-1.3.1-3.el8.x86_64.rpm 140 kB/s | 75 kB 00:00 2023-06-22T17:59:54.834 INFO:teuthology.orchestra.run.smithi039.stdout:(82/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 49 MB/s | 3.7 MB 00:00 2023-06-22T17:59:54.850 INFO:teuthology.orchestra.run.smithi039.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-06-22T17:59:54.871 INFO:teuthology.orchestra.run.smithi039.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 25 MB/s | 538 kB 00:00 2023-06-22T17:59:54.887 INFO:teuthology.orchestra.run.smithi039.stdout:(85/120): socat-1.7.3.3-2.el8.x86_64.rpm 18 MB/s | 302 kB 00:00 2023-06-22T17:59:54.902 INFO:teuthology.orchestra.run.smithi039.stdout:(86/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-06-22T17:59:54.919 INFO:teuthology.orchestra.run.smithi039.stdout:(87/120): thrift-0.13.0-2.el8.x86_64.rpm 2.3 MB/s | 1.7 MB 00:00 2023-06-22T17:59:54.922 INFO:teuthology.orchestra.run.smithi039.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.3 MB/s | 110 kB 00:00 2023-06-22T17:59:54.939 INFO:teuthology.orchestra.run.smithi133.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 137 kB/s | 16 kB 00:00 2023-06-22T17:59:54.940 INFO:teuthology.orchestra.run.smithi133.stdout:(46/120): python3-cachetools-3.1.1-4.el8.noarch 268 kB/s | 33 kB 00:00 2023-06-22T17:59:54.942 INFO:teuthology.orchestra.run.smithi039.stdout:(89/120): python3-jsonpointer-1.10-11.el8.noarc 1.0 MB/s | 20 kB 00:00 2023-06-22T17:59:54.964 INFO:teuthology.orchestra.run.smithi039.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 7.9 MB/s | 353 kB 00:00 2023-06-22T17:59:54.971 INFO:teuthology.orchestra.run.smithi039.stdout:(91/120): libtool-2.4.6-25.el8.x86_64.rpm 24 MB/s | 709 kB 00:00 2023-06-22T17:59:54.977 INFO:teuthology.orchestra.run.smithi039.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.9 MB/s | 54 kB 00:00 2023-06-22T17:59:55.065 INFO:teuthology.orchestra.run.smithi133.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 655 kB/s | 81 kB 00:00 2023-06-22T17:59:55.093 INFO:teuthology.orchestra.run.smithi039.stdout:(93/120): openblas-threads-0.3.12-1.el8.x86_64. 39 MB/s | 4.7 MB 00:00 2023-06-22T17:59:55.114 INFO:teuthology.orchestra.run.smithi039.stdout:(94/120): openblas-0.3.12-1.el8.x86_64.rpm 34 MB/s | 4.6 MB 00:00 2023-06-22T17:59:55.127 INFO:teuthology.orchestra.run.smithi039.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 21 MB/s | 713 kB 00:00 2023-06-22T17:59:55.128 INFO:teuthology.orchestra.run.smithi039.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.3 MB/s | 19 kB 00:00 2023-06-22T17:59:55.141 INFO:teuthology.orchestra.run.smithi039.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.1 MB/s | 43 kB 00:00 2023-06-22T17:59:55.145 INFO:teuthology.orchestra.run.smithi039.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.2 MB/s | 155 kB 00:00 2023-06-22T17:59:55.157 INFO:teuthology.orchestra.run.smithi039.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 14 MB/s | 220 kB 00:00 2023-06-22T17:59:55.158 INFO:teuthology.orchestra.run.smithi039.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 3.6 MB/s | 48 kB 00:00 2023-06-22T17:59:55.171 INFO:teuthology.orchestra.run.smithi039.stdout:(101/120): python3-requests-oauthlib-1.0.0-1.el 3.3 MB/s | 43 kB 00:00 2023-06-22T17:59:55.173 INFO:teuthology.orchestra.run.smithi039.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 14 MB/s | 229 kB 00:00 2023-06-22T17:59:55.185 INFO:teuthology.orchestra.run.smithi133.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 92 kB/s | 11 kB 00:00 2023-06-22T17:59:55.186 INFO:teuthology.orchestra.run.smithi039.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.4 MB/s | 18 kB 00:00 2023-06-22T17:59:55.212 INFO:teuthology.orchestra.run.smithi039.stdout:(104/120): pcre2-devel-10.32-2.el8.x86_64.rpm 24 MB/s | 605 kB 00:00 2023-06-22T17:59:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:(49/120): python3-cheroot-8.5.2-1.el8.noarch.rp 445 kB/s | 173 kB 00:00 2023-06-22T17:59:55.230 INFO:teuthology.orchestra.run.smithi039.stdout:(105/120): openssl-devel-1.1.1g-15.el8_3.x86_64 40 MB/s | 2.3 MB 00:00 2023-06-22T17:59:55.232 INFO:teuthology.orchestra.run.smithi039.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 4.0 MB/s | 83 kB 00:00 2023-06-22T17:59:55.244 INFO:teuthology.orchestra.run.smithi039.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 4.2 MB/s | 59 kB 00:00 2023-06-22T17:59:55.247 INFO:teuthology.orchestra.run.smithi039.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-06-22T17:59:55.259 INFO:teuthology.orchestra.run.smithi039.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.5 MB/s | 38 kB 00:00 2023-06-22T17:59:55.267 INFO:teuthology.orchestra.run.smithi039.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.3 MB/s | 86 kB 00:00 2023-06-22T17:59:55.274 INFO:teuthology.orchestra.run.smithi039.stdout:(111/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.9 MB/s | 58 kB 00:00 2023-06-22T17:59:55.282 INFO:teuthology.orchestra.run.smithi039.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.0 MB/s | 47 kB 00:00 2023-06-22T17:59:55.298 INFO:teuthology.orchestra.run.smithi133.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 156 kB/s | 18 kB 00:00 2023-06-22T17:59:55.301 INFO:teuthology.orchestra.run.smithi039.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 20 MB/s | 559 kB 00:00 2023-06-22T17:59:55.312 INFO:teuthology.orchestra.run.smithi039.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 21 MB/s | 639 kB 00:00 2023-06-22T17:59:55.314 INFO:teuthology.orchestra.run.smithi039.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 5.6 MB/s | 70 kB 00:00 2023-06-22T17:59:55.329 INFO:teuthology.orchestra.run.smithi039.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 12 MB/s | 200 kB 00:00 2023-06-22T17:59:55.330 INFO:teuthology.orchestra.run.smithi039.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 6.0 MB/s | 97 kB 00:00 2023-06-22T17:59:55.394 INFO:teuthology.orchestra.run.smithi133.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 504 kB/s | 49 kB 00:00 2023-06-22T17:59:55.497 INFO:teuthology.orchestra.run.smithi133.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 573 kB/s | 59 kB 00:00 2023-06-22T17:59:55.523 INFO:teuthology.orchestra.run.smithi133.stdout:(53/120): python3-cherrypy-18.4.0-1.el8.noarch. 658 kB/s | 384 kB 00:00 2023-06-22T17:59:55.612 INFO:teuthology.orchestra.run.smithi133.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 525 kB/s | 60 kB 00:00 2023-06-22T17:59:55.636 INFO:teuthology.orchestra.run.smithi133.stdout:(55/120): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2023-06-22T17:59:55.667 INFO:teuthology.orchestra.run.smithi133.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-06-22T17:59:55.700 INFO:teuthology.orchestra.run.smithi133.stdout:(57/120): python3-portend-2.6-1.el8.noarch.rpm 177 kB/s | 16 kB 00:00 2023-06-22T17:59:55.727 INFO:teuthology.orchestra.run.smithi133.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 372 kB/s | 34 kB 00:00 2023-06-22T17:59:55.800 INFO:teuthology.orchestra.run.smithi133.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-06-22T17:59:55.818 INFO:teuthology.orchestra.run.smithi133.stdout:(60/120): python3-rsa-4.8-1.el8.noarch.rpm 524 kB/s | 61 kB 00:00 2023-06-22T17:59:55.830 INFO:teuthology.orchestra.run.smithi133.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 183 kB/s | 19 kB 00:00 2023-06-22T17:59:55.893 INFO:teuthology.orchestra.run.smithi133.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 257 kB/s | 24 kB 00:00 2023-06-22T17:59:55.922 INFO:teuthology.orchestra.run.smithi133.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 280 kB/s | 29 kB 00:00 2023-06-22T17:59:55.929 INFO:teuthology.orchestra.run.smithi133.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 277 kB/s | 27 kB 00:00 2023-06-22T17:59:56.027 INFO:teuthology.orchestra.run.smithi133.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-06-22T17:59:56.030 INFO:teuthology.orchestra.run.smithi133.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 609 kB/s | 61 kB 00:00 2023-06-22T17:59:56.070 INFO:teuthology.orchestra.run.smithi133.stdout:(67/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-06-22T17:59:56.082 INFO:teuthology.orchestra.run.smithi039.stdout:(118/120): librados2-17.2.6-719.g42ddc46e.el8.x 4.6 MB/s | 3.5 MB 00:00 2023-06-22T17:59:56.108 INFO:teuthology.orchestra.run.smithi133.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 296 kB/s | 23 kB 00:00 2023-06-22T17:59:56.112 INFO:teuthology.orchestra.run.smithi133.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 17 MB/s | 70 kB 00:00 2023-06-22T17:59:56.115 INFO:teuthology.orchestra.run.smithi133.stdout:(70/120): python3-webtest-2.0.33-1.el8.noarch.r 969 kB/s | 85 kB 00:00 2023-06-22T17:59:56.140 INFO:teuthology.orchestra.run.smithi039.stdout:(119/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 4.4 MB/s | 3.6 MB 00:00 2023-06-22T17:59:56.147 INFO:teuthology.orchestra.run.smithi133.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 542 kB/s | 19 kB 00:00 2023-06-22T17:59:56.152 INFO:teuthology.orchestra.run.smithi133.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2023-06-22T17:59:56.163 INFO:teuthology.orchestra.run.smithi133.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.4 MB/s | 155 kB 00:00 2023-06-22T17:59:56.169 INFO:teuthology.orchestra.run.smithi133.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-06-22T17:59:56.178 INFO:teuthology.orchestra.run.smithi133.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.3 MB/s | 48 kB 00:00 2023-06-22T17:59:56.186 INFO:teuthology.orchestra.run.smithi133.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-06-22T17:59:56.193 INFO:teuthology.orchestra.run.smithi133.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.9 MB/s | 43 kB 00:00 2023-06-22T17:59:56.205 INFO:teuthology.orchestra.run.smithi133.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.4 MB/s | 18 kB 00:00 2023-06-22T17:59:56.227 INFO:teuthology.orchestra.run.smithi133.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 28 MB/s | 605 kB 00:00 2023-06-22T17:59:56.234 INFO:teuthology.orchestra.run.smithi133.stdout:(80/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 49 MB/s | 2.3 MB 00:00 2023-06-22T17:59:56.243 INFO:teuthology.orchestra.run.smithi133.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 5.3 MB/s | 83 kB 00:00 2023-06-22T17:59:56.249 INFO:teuthology.orchestra.run.smithi133.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.8 MB/s | 59 kB 00:00 2023-06-22T17:59:56.259 INFO:teuthology.orchestra.run.smithi133.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-06-22T17:59:56.263 INFO:teuthology.orchestra.run.smithi133.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.7 MB/s | 38 kB 00:00 2023-06-22T17:59:56.274 INFO:teuthology.orchestra.run.smithi133.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-06-22T17:59:56.276 INFO:teuthology.orchestra.run.smithi133.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.3 MB/s | 58 kB 00:00 2023-06-22T17:59:56.287 INFO:teuthology.orchestra.run.smithi133.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.6 MB/s | 47 kB 00:00 2023-06-22T17:59:56.295 INFO:teuthology.orchestra.run.smithi133.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 30 MB/s | 559 kB 00:00 2023-06-22T17:59:56.306 INFO:teuthology.orchestra.run.smithi133.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 33 MB/s | 639 kB 00:00 2023-06-22T17:59:56.308 INFO:teuthology.orchestra.run.smithi133.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.2 MB/s | 70 kB 00:00 2023-06-22T17:59:56.321 INFO:teuthology.orchestra.run.smithi133.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 13 MB/s | 200 kB 00:00 2023-06-22T17:59:56.322 INFO:teuthology.orchestra.run.smithi133.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.8 MB/s | 97 kB 00:00 2023-06-22T17:59:56.335 INFO:teuthology.orchestra.run.smithi133.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.4 MB/s | 44 kB 00:00 2023-06-22T17:59:56.348 INFO:teuthology.orchestra.run.smithi133.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.1 MB/s | 27 kB 00:00 2023-06-22T17:59:56.361 INFO:teuthology.orchestra.run.smithi133.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 12 MB/s | 161 kB 00:00 2023-06-22T17:59:56.378 INFO:teuthology.orchestra.run.smithi133.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 20 MB/s | 320 kB 00:00 2023-06-22T17:59:56.383 INFO:teuthology.orchestra.run.smithi133.stdout:(97/120): thrift-0.13.0-2.el8.x86_64.rpm 5.5 MB/s | 1.7 MB 00:00 2023-06-22T17:59:56.401 INFO:teuthology.orchestra.run.smithi133.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 20 MB/s | 457 kB 00:00 2023-06-22T17:59:56.414 INFO:teuthology.orchestra.run.smithi133.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.9 MB/s | 39 kB 00:00 2023-06-22T17:59:56.436 INFO:teuthology.orchestra.run.smithi133.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 30 MB/s | 688 kB 00:00 2023-06-22T17:59:56.454 INFO:teuthology.orchestra.run.smithi133.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-06-22T17:59:56.481 INFO:teuthology.orchestra.run.smithi133.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 49 MB/s | 4.8 MB 00:00 2023-06-22T17:59:56.482 INFO:teuthology.orchestra.run.smithi133.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 7.7 MB/s | 225 kB 00:00 2023-06-22T17:59:56.496 INFO:teuthology.orchestra.run.smithi133.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.4 MB/s | 126 kB 00:00 2023-06-22T17:59:56.512 INFO:teuthology.orchestra.run.smithi133.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-06-22T17:59:56.630 INFO:teuthology.orchestra.run.smithi133.stdout:(106/120): python3-scipy-1.0.0-20.module+el8.1. 47 MB/s | 15 MB 00:00 2023-06-22T17:59:56.638 INFO:teuthology.orchestra.run.smithi133.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 4.2 MB/s | 538 kB 00:00 2023-06-22T17:59:56.646 INFO:teuthology.orchestra.run.smithi133.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 19 MB/s | 302 kB 00:00 2023-06-22T17:59:56.652 INFO:teuthology.orchestra.run.smithi133.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-06-22T17:59:56.662 INFO:teuthology.orchestra.run.smithi133.stdout:(110/120): python3-numpy-1.14.3-9.el8.x86_64.rp 21 MB/s | 3.7 MB 00:00 2023-06-22T17:59:56.664 INFO:teuthology.orchestra.run.smithi133.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 6.2 MB/s | 110 kB 00:00 2023-06-22T17:59:56.668 INFO:teuthology.orchestra.run.smithi133.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 22 MB/s | 353 kB 00:00 2023-06-22T17:59:56.678 INFO:teuthology.orchestra.run.smithi133.stdout:(113/120): python3-jsonpointer-1.10-11.el8.noar 1.3 MB/s | 20 kB 00:00 2023-06-22T17:59:56.684 INFO:teuthology.orchestra.run.smithi133.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 35 MB/s | 709 kB 00:00 2023-06-22T17:59:56.687 INFO:teuthology.orchestra.run.smithi133.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 2.9 MB/s | 54 kB 00:00 2023-06-22T17:59:56.711 INFO:teuthology.orchestra.run.smithi133.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 29 MB/s | 713 kB 00:00 2023-06-22T17:59:56.756 INFO:teuthology.orchestra.run.smithi133.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 60 MB/s | 4.7 MB 00:00 2023-06-22T17:59:56.772 INFO:teuthology.orchestra.run.smithi133.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 52 MB/s | 4.6 MB 00:00 2023-06-22T17:59:57.355 INFO:teuthology.orchestra.run.smithi133.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 5.4 MB/s | 3.5 MB 00:00 2023-06-22T17:59:57.409 INFO:teuthology.orchestra.run.smithi133.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 5.5 MB/s | 3.6 MB 00:00 2023-06-22T17:59:57.414 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:59:57.415 INFO:teuthology.orchestra.run.smithi133.stdout:Total 12 MB/s | 201 MB 00:16 2023-06-22T17:59:57.461 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-06-22T17:59:57.572 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-06-22T17:59:57.573 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-06-22T17:59:57.755 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-06-22T17:59:57.965 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-22T17:59:58.743 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T17:59:58.852 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T17:59:59.038 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-06-22T17:59:59.042 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-06-22T17:59:59.134 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T17:59:59.214 INFO:teuthology.orchestra.run.smithi039.stdout:(120/120): ceph-test-17.2.6-719.g42ddc46e.el8.x 5.8 MB/s | 52 MB 00:08 2023-06-22T17:59:59.218 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2023-06-22T17:59:59.218 INFO:teuthology.orchestra.run.smithi039.stdout:Total 13 MB/s | 201 MB 00:16 2023-06-22T17:59:59.294 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2023-06-22T17:59:59.429 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2023-06-22T17:59:59.430 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2023-06-22T17:59:59.501 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T17:59:59.743 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:00:00.180 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:00:00.409 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T18:00:00.540 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T18:00:00.776 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:00:00.884 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:00:01.040 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2023-06-22T18:00:01.045 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2023-06-22T18:00:01.089 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:00:01.447 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-22T18:00:02.142 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-22T18:00:02.359 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:00:02.458 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:00:02.705 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-22T18:00:02.952 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-22T18:00:03.192 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:00:03.302 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:00:03.517 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T18:00:03.598 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T18:00:04.285 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-22T18:00:04.421 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-06-22T18:00:04.591 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-22T18:00:04.675 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-22T18:00:04.857 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-22T18:00:04.922 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T18:00:05.015 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-22T18:00:05.042 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-22T18:00:05.242 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T18:00:05.272 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T18:00:05.384 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T18:00:05.592 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T18:00:05.651 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-22T18:00:05.905 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:00:06.132 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T18:00:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2023-06-22T18:00:06.303 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:00:06.318 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T18:00:06.348 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-22T18:00:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T18:00:06.558 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-06-22T18:00:06.665 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T18:00:06.765 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-06-22T18:00:06.873 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T18:00:06.900 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-06-22T18:00:06.901 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:00:06.993 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:00:07.131 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-06-22T18:00:07.133 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T18:00:07.227 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:00:07.550 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-22T18:00:07.597 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-06-22T18:00:07.842 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T18:00:07.965 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T18:00:08.153 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-22T18:00:08.252 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-22T18:00:08.451 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-22T18:00:08.462 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-22T18:00:08.665 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:00:08.671 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:00:08.710 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-22T18:00:08.774 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:00:08.826 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:00:08.932 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-22T18:00:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:00:09.020 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-22T18:00:09.108 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-22T18:00:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:00:09.241 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-22T18:00:09.349 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T18:00:09.413 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:00:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-22T18:00:09.566 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:00:09.697 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T18:00:09.800 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-22T18:00:09.814 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T18:00:09.987 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-22T18:00:10.097 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-22T18:00:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-22T18:00:10.293 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-22T18:00:10.366 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T18:00:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:00:10.545 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-22T18:00:10.621 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:00:10.738 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-22T18:00:10.847 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-22T18:00:10.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T18:00:11.050 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-22T18:00:11.079 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-22T18:00:11.248 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:00:11.279 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T18:00:11.384 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:00:11.547 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T18:00:11.648 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-22T18:00:11.793 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-06-22T18:00:11.812 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-22T18:00:11.948 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-06-22T18:00:11.988 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T18:00:12.124 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-06-22T18:00:12.305 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-06-22T18:00:12.384 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T18:00:12.550 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-22T18:00:12.746 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-22T18:00:12.779 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T18:00:12.958 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-06-22T18:00:13.114 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-06-22T18:00:13.498 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-22T18:00:13.751 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-06-22T18:00:13.812 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-22T18:00:13.815 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-22T18:00:13.914 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-06-22T18:00:14.097 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-22T18:00:14.152 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-22T18:00:14.372 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-22T18:00:14.462 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-22T18:00:14.552 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-22T18:00:14.679 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-22T18:00:14.731 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-22T18:00:14.824 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-06-22T18:00:14.860 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-22T18:00:15.028 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T18:00:15.100 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-06-22T18:00:15.238 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-22T18:00:15.309 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-22T18:00:15.310 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-06-22T18:00:15.489 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-06-22T18:00:15.602 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-22T18:00:15.612 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-22T18:00:15.779 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-06-22T18:00:15.853 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T18:00:15.865 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-22T18:00:16.095 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-22T18:00:16.110 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-06-22T18:00:16.323 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-22T18:00:16.366 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-22T18:00:16.564 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-22T18:00:16.572 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-22T18:00:16.711 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-22T18:00:16.827 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-22T18:00:16.958 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T18:00:17.074 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T18:00:17.152 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T18:00:17.374 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-22T18:00:17.532 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-22T18:00:17.749 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-22T18:00:17.868 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-22T18:00:17.961 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-22T18:00:18.100 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-22T18:00:18.157 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-06-22T18:00:18.232 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-22T18:00:18.318 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-22T18:00:18.471 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-22T18:00:18.522 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-06-22T18:00:18.740 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-22T18:00:18.829 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-06-22T18:00:18.969 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-22T18:00:19.095 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-06-22T18:00:19.235 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-22T18:00:19.350 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-22T18:00:19.419 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-06-22T18:00:19.630 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-06-22T18:00:19.662 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-22T18:00:19.781 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-22T18:00:19.971 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-22T18:00:20.081 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-22T18:00:20.267 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-22T18:00:20.292 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-22T18:00:20.483 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-22T18:00:20.506 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-22T18:00:20.715 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-22T18:00:20.787 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-22T18:00:20.967 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-22T18:00:21.024 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-22T18:00:21.240 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-22T18:00:21.337 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-22T18:00:21.414 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-22T18:00:21.584 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-22T18:00:21.628 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-22T18:00:21.818 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-22T18:00:21.827 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-22T18:00:22.021 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-22T18:00:22.107 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-22T18:00:22.239 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-06-22T18:00:22.273 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-22T18:00:22.329 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-22T18:00:22.503 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T18:00:22.504 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-22T18:00:22.586 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-06-22T18:00:22.607 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-22T18:00:22.727 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-22T18:00:22.795 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-22T18:00:22.832 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-22T18:00:22.889 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-22T18:00:23.004 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-22T18:00:23.035 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-22T18:00:23.160 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-22T18:00:23.228 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-22T18:00:23.231 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-22T18:00:23.374 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-22T18:00:23.420 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-22T18:00:23.428 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-22T18:00:23.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-22T18:00:23.606 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-22T18:00:23.663 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-22T18:00:23.778 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-06-22T18:00:23.780 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-22T18:00:23.916 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-22T18:00:23.985 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-06-22T18:00:23.995 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-22T18:00:24.152 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-22T18:00:24.175 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-06-22T18:00:24.281 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-22T18:00:24.357 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-06-22T18:00:24.381 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-22T18:00:24.504 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-06-22T18:00:24.517 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-22T18:00:24.699 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-06-22T18:00:24.737 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-22T18:00:24.742 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-22T18:00:24.877 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-06-22T18:00:25.038 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-22T18:00:25.234 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-06-22T18:00:25.357 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-22T18:00:25.546 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-06-22T18:00:25.667 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-22T18:00:25.868 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-22T18:00:25.870 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-06-22T18:00:25.909 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-22T18:00:26.047 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-22T18:00:26.239 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-06-22T18:00:26.278 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-22T18:00:26.355 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-22T18:00:26.480 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-06-22T18:00:26.498 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-22T18:00:26.568 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-22T18:00:26.656 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-06-22T18:00:26.767 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-22T18:00:26.821 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-22T18:00:26.859 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-06-22T18:00:26.965 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-22T18:00:27.062 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-06-22T18:00:27.104 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-22T18:00:27.142 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-22T18:00:27.237 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-06-22T18:00:27.336 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-22T18:00:27.342 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-06-22T18:00:27.452 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-22T18:00:27.565 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-06-22T18:00:27.581 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-22T18:00:27.733 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-06-22T18:00:27.764 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-22T18:00:27.819 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-22T18:00:27.924 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-22T18:00:27.973 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-22T18:00:28.022 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-22T18:00:28.117 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-22T18:00:28.206 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-22T18:00:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-22T18:00:28.403 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-22T18:00:28.503 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-22T18:00:28.648 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-22T18:00:28.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-22T18:00:28.863 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-22T18:00:28.992 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-22T18:00:29.018 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-22T18:00:29.194 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-22T18:00:29.280 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-22T18:00:29.416 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-22T18:00:29.632 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-22T18:00:29.644 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-22T18:00:29.830 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-22T18:00:29.887 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-22T18:00:30.025 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-22T18:00:30.123 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-22T18:00:30.187 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-22T18:00:30.459 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-22T18:00:30.696 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-22T18:00:30.904 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-22T18:00:31.242 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-22T18:00:31.316 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-22T18:00:31.523 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-22T18:00:31.648 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-22T18:00:31.744 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-22T18:00:31.902 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-22T18:00:32.027 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-22T18:00:32.249 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-22T18:00:32.602 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-22T18:00:32.981 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-22T18:00:33.108 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:33.316 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-22T18:00:33.355 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:33.501 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-22T18:00:33.546 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-22T18:00:33.776 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-22T18:00:33.880 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-22T18:00:33.951 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:34.122 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-22T18:00:34.257 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T18:00:34.367 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T18:00:34.467 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-22T18:00:34.528 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T18:00:34.708 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-22T18:00:35.059 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-22T18:00:35.314 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-22T18:00:35.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-22T18:00:35.703 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-22T18:00:38.856 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:39.141 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:39.751 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T18:00:40.764 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T18:00:40.924 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T18:00:42.761 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:43.002 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:44.417 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-22T18:00:44.674 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T18:00:44.902 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-22T18:00:45.043 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T18:00:52.259 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T18:00:52.259 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /sys 2023-06-22T18:00:52.259 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /proc 2023-06-22T18:00:52.259 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-06-22T18:00:52.259 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /run 2023-06-22T18:00:52.260 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /mnt 2023-06-22T18:00:52.260 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /var/tmp 2023-06-22T18:00:52.260 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /home 2023-06-22T18:00:52.260 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /tmp 2023-06-22T18:00:52.260 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-06-22T18:00:52.260 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T18:00:52.661 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T18:00:54.148 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T18:00:54.346 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T18:00:54.567 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T18:00:54.719 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T18:00:55.105 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T18:00:55.269 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T18:00:55.285 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T18:00:57.986 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T18:00:58.241 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T18:00:58.400 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-22T18:00:58.600 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-22T18:00:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T18:00:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2023-06-22T18:00:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2023-06-22T18:00:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-06-22T18:00:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /run 2023-06-22T18:00:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2023-06-22T18:00:58.700 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2023-06-22T18:00:58.701 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2023-06-22T18:00:58.701 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2023-06-22T18:00:58.701 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-06-22T18:00:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:00:58.784 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T18:00:59.000 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T18:00:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T18:00:59.254 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T18:01:00.214 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T18:01:00.443 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T18:01:00.541 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T18:01:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T18:01:00.764 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T18:01:00.933 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-22T18:01:00.983 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T18:01:01.154 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T18:01:01.190 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T18:01:01.213 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T18:01:01.451 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T18:01:01.462 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T18:01:01.600 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T18:01:01.779 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T18:01:01.795 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T18:01:03.061 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-22T18:01:03.303 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T18:01:03.417 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T18:01:03.601 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T18:01:03.644 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T18:01:03.859 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T18:01:04.205 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T18:01:04.401 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T18:01:04.688 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T18:01:04.950 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /run 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2023-06-22T18:01:04.951 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2023-06-22T18:01:04.952 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2023-06-22T18:01:04.952 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2023-06-22T18:01:04.952 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:01:04.957 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T18:01:05.059 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-22T18:01:05.292 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-22T18:01:05.382 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T18:01:05.494 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T18:01:05.707 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T18:01:05.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T18:01:06.926 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T18:01:07.128 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-22T18:01:07.153 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T18:01:07.249 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T18:01:07.312 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T18:01:07.601 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-22T18:01:07.651 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-22T18:01:07.757 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T18:01:07.829 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T18:01:07.865 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T18:01:08.069 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T18:01:08.080 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T18:01:08.243 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-22T18:01:08.406 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T18:01:08.448 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-22T18:01:09.304 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T18:01:09.706 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-22T18:01:09.946 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T18:01:10.039 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T18:01:10.228 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T18:01:10.242 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-22T18:01:10.270 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T18:01:10.519 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T18:01:10.643 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-22T18:01:10.853 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-22T18:01:10.864 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T18:01:11.077 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T18:01:11.115 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-22T18:01:11.333 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T18:01:11.344 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T18:01:11.480 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-22T18:01:11.671 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-22T18:01:11.800 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-22T18:01:12.014 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-22T18:01:12.312 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-22T18:01:12.550 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T18:01:12.609 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T18:01:12.810 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T18:01:12.816 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T18:01:12.825 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T18:01:12.960 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-22T18:01:12.961 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T18:01:12.982 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-22T18:01:13.189 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-22T18:01:13.194 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T18:01:13.194 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-22T18:01:13.555 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T18:01:13.739 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:13.739 INFO:teuthology.orchestra.run.smithi101.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-22T18:01:13.739 INFO:teuthology.orchestra.run.smithi101.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-22T18:01:13.739 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T18:01:13.838 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-22T18:01:14.011 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T18:01:14.065 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-22T18:01:14.065 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-22T18:01:14.066 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-22T18:01:14.067 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-22T18:01:14.068 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-22T18:01:14.068 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-22T18:01:14.068 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-22T18:01:14.068 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-22T18:01:14.068 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-22T18:01:14.068 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T18:01:14.069 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-22T18:01:14.070 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-22T18:01:14.071 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-22T18:01:14.072 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-22T18:01:14.073 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-22T18:01:14.074 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-22T18:01:14.075 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-22T18:01:14.076 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-22T18:01:14.077 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-22T18:01:14.078 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-22T18:01:14.079 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-22T18:01:14.080 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-22T18:01:14.081 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-22T18:01:15.215 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-22T18:01:15.397 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T18:01:15.546 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-22T18:01:15.957 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-22T18:01:15.978 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T18:01:16.169 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T18:01:16.241 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-22T18:01:16.409 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:16.409 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:46 ago on Thu 22 Jun 2023 05:59:30 PM UTC. 2023-06-22T18:01:16.487 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T18:01:16.516 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-22T18:01:16.946 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-22T18:01:17.396 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-22T18:01:17.571 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-22T18:01:17.816 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-22T18:01:18.026 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T18:01:18.037 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-22T18:01:18.257 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-22T18:01:18.355 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-22T18:01:18.414 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-22T18:01:18.416 INFO:teuthology.orchestra.run.smithi101.stdout:Installed products updated. 2023-06-22T18:01:18.538 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T18:01:18.538 INFO:teuthology.orchestra.run.smithi101.stdout:Upgraded: 2023-06-22T18:01:18.538 INFO:teuthology.orchestra.run.smithi101.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.538 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.538 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-06-22T18:01:18.538 INFO:teuthology.orchestra.run.smithi101.stdout: automake-1.16.1-7.el8.noarch 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: bison-3.0.4-10.el8.x86_64 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.539 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.540 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: flex-2.6.1-9.el8.x86_64 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-22T18:01:18.542 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: jq-1.5-12.el8.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.543 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.544 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-22T18:01:18.545 INFO:teuthology.orchestra.run.smithi101.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-22T18:01:18.546 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.547 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-22T18:01:18.548 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-22T18:01:18.549 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend-2.6-1.el8.noarch 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-22T18:01:18.550 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.551 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-22T18:01:18.552 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.553 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:18.554 INFO:teuthology.orchestra.run.smithi101.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-22T18:01:18.554 INFO:teuthology.orchestra.run.smithi101.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-22T18:01:18.554 INFO:teuthology.orchestra.run.smithi101.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-22T18:01:18.554 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-22T18:01:18.554 INFO:teuthology.orchestra.run.smithi101.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-22T18:01:18.555 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-06-22T18:01:18.555 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-06-22T18:01:18.569 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T18:01:18.609 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-22T18:01:18.697 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-22T18:01:18.732 DEBUG:teuthology.parallel:result is None 2023-06-22T18:01:18.830 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-22T18:01:18.863 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T18:01:18.938 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-22T18:01:19.091 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-22T18:01:19.288 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T18:01:19.334 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T18:01:19.392 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-22T18:01:19.542 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T18:01:19.550 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-22T18:01:19.703 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-22T18:01:19.703 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T18:01:19.727 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-22T18:01:19.912 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T18:01:19.953 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T18:01:19.954 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:20.482 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:20.483 INFO:teuthology.orchestra.run.smithi133.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-22T18:01:20.483 INFO:teuthology.orchestra.run.smithi133.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-22T18:01:20.483 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:01:20.800 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-22T18:01:20.801 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-22T18:01:20.801 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-22T18:01:20.801 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:01:20.802 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-22T18:01:20.802 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:01:20.802 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:01:20.802 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-22T18:01:20.803 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-22T18:01:20.803 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:01:20.803 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-22T18:01:20.803 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-22T18:01:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:01:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-22T18:01:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-22T18:01:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-22T18:01:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-22T18:01:20.804 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-22T18:01:20.805 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-22T18:01:20.806 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-22T18:01:20.806 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T18:01:20.812 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-22T18:01:20.812 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-22T18:01:20.812 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-22T18:01:20.812 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-22T18:01:20.812 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-22T18:01:20.812 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-22T18:01:20.813 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-22T18:01:20.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-22T18:01:20.815 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-22T18:01:20.816 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-22T18:01:20.816 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-22T18:01:20.816 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-22T18:01:20.816 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-22T18:01:20.816 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-22T18:01:20.816 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-22T18:01:20.817 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-22T18:01:20.818 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-22T18:01:20.819 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-22T18:01:20.819 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-22T18:01:20.819 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-22T18:01:20.819 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-22T18:01:20.819 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-22T18:01:20.820 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-22T18:01:20.821 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-22T18:01:20.822 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-22T18:01:20.823 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-22T18:01:20.823 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-22T18:01:20.823 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-22T18:01:20.823 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-22T18:01:20.823 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-22T18:01:20.824 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-22T18:01:20.824 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-22T18:01:20.824 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-22T18:01:20.824 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-22T18:01:20.824 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-22T18:01:20.824 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-22T18:01:20.825 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-22T18:01:20.825 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-22T18:01:20.825 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-22T18:01:20.825 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-22T18:01:23.131 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:23.131 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:52 ago on Thu 22 Jun 2023 05:59:31 PM UTC. 2023-06-22T18:01:25.083 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-06-22T18:01:25.203 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:01:25.204 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2023-06-22T18:01:25.204 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.204 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.204 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-06-22T18:01:25.204 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.1-7.el8.noarch 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.0.4-10.el8.x86_64 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.205 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.206 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.207 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.1-9.el8.x86_64 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.5-12.el8.x86_64 2023-06-22T18:01:25.208 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-22T18:01:25.209 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-22T18:01:25.210 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-22T18:01:25.211 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-22T18:01:25.212 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-22T18:01:25.213 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-22T18:01:25.214 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-22T18:01:25.215 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-2.6-1.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-22T18:01:25.216 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-22T18:01:25.217 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-22T18:01:25.218 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-22T18:01:25.219 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-22T18:01:25.219 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-22T18:01:25.219 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.219 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.219 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:25.220 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-22T18:01:25.220 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-22T18:01:25.220 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-22T18:01:25.220 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-22T18:01:25.220 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-22T18:01:25.221 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:01:25.221 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-06-22T18:01:25.419 DEBUG:teuthology.parallel:result is None 2023-06-22T18:01:25.697 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-22T18:01:26.168 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-22T18:01:26.328 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-06-22T18:01:27.504 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-06-22T18:01:27.867 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-06-22T18:01:28.053 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 111/122 2023-06-22T18:01:28.224 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-06-22T18:01:28.409 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-22T18:01:28.692 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-22T18:01:28.854 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-22T18:01:29.036 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-22T18:01:29.188 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-22T18:01:29.419 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-06-22T18:01:29.602 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-06-22T18:01:29.679 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-06-22T18:01:29.836 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-06-22T18:01:29.860 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-06-22T18:01:29.861 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T18:01:30.142 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-22T18:01:30.143 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:30.495 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:30.495 INFO:teuthology.orchestra.run.smithi039.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-06-22T18:01:30.495 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:01:30.962 INFO:teuthology.orchestra.run.smithi039.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-22T18:01:30.962 INFO:teuthology.orchestra.run.smithi039.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-22T18:01:30.963 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:01:31.363 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-22T18:01:31.363 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-22T18:01:31.364 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-22T18:01:31.364 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-22T18:01:31.364 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-22T18:01:31.364 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-22T18:01:31.364 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-22T18:01:31.365 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-22T18:01:31.366 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-22T18:01:31.367 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-22T18:01:31.367 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-22T18:01:31.367 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-22T18:01:31.374 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-22T18:01:31.374 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-22T18:01:31.374 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-22T18:01:31.374 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-22T18:01:31.375 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-22T18:01:31.376 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-22T18:01:31.377 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-22T18:01:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-22T18:01:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-22T18:01:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-22T18:01:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-22T18:01:31.378 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-22T18:01:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-22T18:01:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-22T18:01:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-22T18:01:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-22T18:01:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-22T18:01:31.379 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-22T18:01:31.380 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-22T18:01:31.380 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-22T18:01:31.380 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-22T18:01:31.380 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-22T18:01:31.380 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-22T18:01:31.380 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-22T18:01:31.381 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-22T18:01:31.381 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-22T18:01:31.381 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-22T18:01:31.381 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-22T18:01:31.381 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-06-22T18:01:31.381 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-06-22T18:01:31.382 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-06-22T18:01:31.383 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-06-22T18:01:31.384 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-06-22T18:01:31.385 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-06-22T18:01:31.385 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-06-22T18:01:31.385 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-06-22T18:01:31.385 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-06-22T18:01:31.385 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-06-22T18:01:31.385 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-06-22T18:01:31.386 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-06-22T18:01:31.387 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-06-22T18:01:31.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-06-22T18:01:31.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-22T18:01:31.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-22T18:01:31.388 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-22T18:01:33.705 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-22T18:01:33.706 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:00 ago on Thu 22 Jun 2023 05:59:33 PM UTC. 2023-06-22T18:01:35.897 INFO:teuthology.orchestra.run.smithi039.stdout:Installed products updated. 2023-06-22T18:01:36.035 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:01:36.036 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2023-06-22T18:01:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.036 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2023-06-22T18:01:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.1-7.el8.noarch 2023-06-22T18:01:36.036 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.0.4-10.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.037 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.038 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.039 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.1-9.el8.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: jq-1.5-12.el8.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-22T18:01:36.040 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-22T18:01:36.041 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-22T18:01:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-22T18:01:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-22T18:01:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.042 INFO:teuthology.orchestra.run.smithi039.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-22T18:01:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-22T18:01:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-22T18:01:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-22T18:01:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-22T18:01:36.043 INFO:teuthology.orchestra.run.smithi039.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-22T18:01:36.044 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.045 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-22T18:01:36.046 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-22T18:01:36.047 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-22T18:01:36.048 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-2.6-1.el8.noarch 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-22T18:01:36.049 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-22T18:01:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-22T18:01:36.051 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-22T18:01:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-22T18:01:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-22T18:01:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-22T18:01:36.052 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-22T18:01:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-22T18:01:36.053 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:01:36.053 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2023-06-22T18:01:36.330 DEBUG:teuthology.parallel:result is None 2023-06-22T18:01:36.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T18:01:36.465 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-22T18:01:36.512 INFO:teuthology.orchestra.run.smithi039.stdout:17.2.6-719.g42ddc46e.el8 2023-06-22T18:01:36.513 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-22T18:01:36.513 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-22T18:01:36.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T18:01:36.652 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-22T18:01:36.677 INFO:teuthology.orchestra.run.smithi101.stdout:17.2.6-719.g42ddc46e.el8 2023-06-22T18:01:36.677 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-22T18:01:36.678 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-22T18:01:36.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T18:01:36.890 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-22T18:01:36.915 INFO:teuthology.orchestra.run.smithi133.stdout:17.2.6-719.g42ddc46e.el8 2023-06-22T18:01:36.916 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-22T18:01:36.916 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-22T18:01:36.917 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-22T18:01:36.918 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:01:36.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-22T18:01:37.003 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:01:37.004 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-22T18:01:37.046 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:01:37.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-22T18:01:37.089 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-22T18:01:37.090 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:01:37.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2023-06-22T18:01:37.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-22T18:01:37.258 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:01:37.259 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/daemon-helper 2023-06-22T18:01:37.287 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-22T18:01:37.355 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:01:37.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2023-06-22T18:01:37.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-22T18:01:37.452 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-22T18:01:37.453 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:01:37.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-22T18:01:37.514 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-22T18:01:37.614 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:01:37.614 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-22T18:01:37.642 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-22T18:01:37.710 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:01:37.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-22T18:01:37.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-22T18:01:37.806 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-22T18:01:37.806 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:01:37.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2023-06-22T18:01:37.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-22T18:01:37.970 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:01:37.970 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/stdin-killer 2023-06-22T18:01:37.998 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-22T18:01:38.065 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:01:38.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2023-06-22T18:01:38.093 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-22T18:01:38.161 INFO:teuthology.run_tasks:Running task cephadm... 2023-06-22T18:01:38.247 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-06-22T18:01:38.248 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-22T18:01:38.257 INFO:tasks.cephadm:Cluster fsid is d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:01:38.258 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-06-22T18:01:38.259 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.101', 'mon.c': '172.21.15.133'} 2023-06-22T18:01:38.259 INFO:tasks.cephadm:First mon is mon.a on smithi039 2023-06-22T18:01:38.260 INFO:tasks.cephadm:First mgr is x 2023-06-22T18:01:38.260 INFO:tasks.cephadm:Normalizing hostnames... 2023-06-22T18:01:38.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2023-06-22T18:01:38.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname $(hostname -s) 2023-06-22T18:01:38.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2023-06-22T18:01:38.383 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph-ci.git ref 42ddc46e6e6582d8e29a5a982a81886fe028be2c)... 2023-06-22T18:01:38.384 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/42ddc46e6e6582d8e29a5a982a81886fe028be2c/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:38.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366215 Jun 22 18:01 /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:38.692 DEBUG:teuthology.orchestra.run.smithi101:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/42ddc46e6e6582d8e29a5a982a81886fe028be2c/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:38.852 INFO:teuthology.orchestra.run.smithi101.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366215 Jun 22 18:01 /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:38.853 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/42ddc46e6e6582d8e29a5a982a81886fe028be2c/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:39.043 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-rw-r--. 1 ubuntu ubuntu 366215 Jun 22 18:01 /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:39.044 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:39.087 DEBUG:teuthology.orchestra.run.smithi101:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:39.106 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-22T18:01:39.138 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c on all hosts... 2023-06-22T18:01:39.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c pull 2023-06-22T18:01:39.143 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c pull 2023-06-22T18:01:39.148 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c pull 2023-06-22T18:01:42.962 INFO:teuthology.orchestra.run.smithi101.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-22T18:01:44.093 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-22T18:01:44.541 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-22T18:02:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2023-06-22T18:02:31.046 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable)", 2023-06-22T18:02:31.046 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710", 2023-06-22T18:02:31.047 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2023-06-22T18:02:31.047 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b9658f1f8fdc87272ce6f577d5ecedf811fa77b9a03555eb6b1de17c25ce4236" 2023-06-22T18:02:31.047 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2023-06-22T18:02:31.047 INFO:teuthology.orchestra.run.smithi039.stdout:} 2023-06-22T18:02:39.351 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-06-22T18:02:39.352 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_version": "ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable)", 2023-06-22T18:02:39.352 INFO:teuthology.orchestra.run.smithi101.stdout: "image_id": "f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710", 2023-06-22T18:02:39.352 INFO:teuthology.orchestra.run.smithi101.stdout: "repo_digests": [ 2023-06-22T18:02:39.352 INFO:teuthology.orchestra.run.smithi101.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b9658f1f8fdc87272ce6f577d5ecedf811fa77b9a03555eb6b1de17c25ce4236" 2023-06-22T18:02:39.352 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-06-22T18:02:39.353 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-06-22T18:02:40.419 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-06-22T18:02:40.419 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable)", 2023-06-22T18:02:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710", 2023-06-22T18:02:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2023-06-22T18:02:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:b9658f1f8fdc87272ce6f577d5ecedf811fa77b9a03555eb6b1de17c25ce4236" 2023-06-22T18:02:40.420 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-06-22T18:02:40.420 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-06-22T18:02:40.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2023-06-22T18:02:40.519 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /etc/ceph 2023-06-22T18:02:40.552 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2023-06-22T18:02:40.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2023-06-22T18:02:40.647 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 777 /etc/ceph 2023-06-22T18:02:40.676 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2023-06-22T18:02:40.704 INFO:tasks.cephadm:Writing seed config... 2023-06-22T18:02:40.706 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-06-22T18:02:40.706 INFO:tasks.cephadm: override: [client] debug client = 20 2023-06-22T18:02:40.706 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-06-22T18:02:40.707 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-06-22T18:02:40.707 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-06-22T18:02:40.707 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-06-22T18:02:40.708 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-06-22T18:02:40.708 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-06-22T18:02:40.709 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-06-22T18:02:40.709 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-06-22T18:02:40.709 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-06-22T18:02:40.710 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-06-22T18:02:40.710 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-06-22T18:02:40.710 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-06-22T18:02:40.710 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-06-22T18:02:40.711 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-06-22T18:02:40.711 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-06-22T18:02:40.711 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-06-22T18:02:40.712 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-06-22T18:02:40.712 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-06-22T18:02:40.712 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-06-22T18:02:40.713 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-06-22T18:02:40.713 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-06-22T18:02:40.714 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-06-22T18:02:40.714 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-06-22T18:02:40.714 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-06-22T18:02:40.715 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-06-22T18:02:40.715 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-06-22T18:02:40.715 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-06-22T18:02:40.716 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-06-22T18:02:40.716 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-06-22T18:02:40.717 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-06-22T18:02:40.717 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-06-22T18:02:40.717 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-06-22T18:02:40.718 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-06-22T18:02:40.718 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-06-22T18:02:40.719 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-06-22T18:02:40.719 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-06-22T18:02:40.719 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-06-22T18:02:40.720 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-06-22T18:02:40.720 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-06-22T18:02:40.721 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:02:40.722 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-06-22T18:02:40.759 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d572fea0-1126-11ee-9b2b-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-06-22T18:02:40.760 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a.service 2023-06-22T18:02:40.803 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x.service 2023-06-22T18:02:40.847 INFO:tasks.cephadm:Bootstrapping... 2023-06-22T18:02:40.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c -v bootstrap --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:02:40.863 INFO:journalctl@ceph.mon.a.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:02:40.910 INFO:journalctl@ceph.mgr.x.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:02:41.119 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2023-06-22T18:02:41.119 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c', '-v', 'bootstrap', '--fsid', 'd572fea0-1126-11ee-9b2b-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2023-06-22T18:02:41.290 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 3.2.3 2023-06-22T18:02:41.290 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-06-22T18:02:41.291 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2023-06-22T18:02:41.458 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 3.2.3 2023-06-22T18:02:41.459 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2023-06-22T18:02:41.459 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2023-06-22T18:02:41.473 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-22T18:02:41.473 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-22T18:02:41.486 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-22T18:02:41.486 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-06-22T18:02:41.500 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2023-06-22T18:02:41.513 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2023-06-22T18:02:41.513 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2023-06-22T18:02:41.513 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2023-06-22T18:02:41.682 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 3.2.3 2023-06-22T18:02:41.683 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 3.2.3 is present 2023-06-22T18:02:41.683 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2023-06-22T18:02:41.683 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2023-06-22T18:02:41.696 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-22T18:02:41.696 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-22T18:02:41.708 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-22T18:02:41.709 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-06-22T18:02:41.722 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2023-06-22T18:02:41.735 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2023-06-22T18:02:41.735 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2023-06-22T18:02:41.736 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2023-06-22T18:02:41.736 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:02:41.736 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 140059361962472 on /run/cephadm/d572fea0-1126-11ee-9b2b-001a4aab830c.lock 2023-06-22T18:02:41.736 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 140059361962472 acquired on /run/cephadm/d572fea0-1126-11ee-9b2b-001a4aab830c.lock 2023-06-22T18:02:41.737 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2023-06-22T18:02:41.737 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2023-06-22T18:02:41.737 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2023-06-22T18:02:41.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-06-22T18:02:41.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2023-06-22T18:02:41.751 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-06-22T18:02:41.751 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-06-22T18:02:41.751 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2023-06-22T18:02:41.757 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-06-22T18:02:41.757 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-06-22T18:02:41.757 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-22T18:02:41.757 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-06-22T18:02:41.757 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link 2023-06-22T18:02:41.757 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-22T18:02:41.758 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2023-06-22T18:02:41.758 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2023-06-22T18:02:41.758 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-06-22T18:02:41.759 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-06-22T18:02:41.760 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-22T18:02:42.328 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710 2023-06-22T18:02:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c... 2023-06-22T18:02:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2023-06-22T18:02:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:9d60a6994110d194fea96cb2ff67b650891faff48e39f8a6b8ad8152d347e90d 2023-06-22T18:02:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:3dbfccf0662dd88e0c2efa9d59916ea5c98f7c5a05bbe18e86c2dc32f0494158 2023-06-22T18:02:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:f3276efde470d44ef0ff8e259464d1d34a5015beb9d7c1215520bcf3b48fd710 2023-06-22T18:02:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2023-06-22T18:02:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Storing signatures 2023-06-22T18:02:43.198 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable) 2023-06-22T18:02:43.199 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable) 2023-06-22T18:02:43.199 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2023-06-22T18:02:43.971 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2023-06-22T18:02:43.971 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2023-06-22T18:02:44.745 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQDEjJRkK1ssFhAAQw0hOGwl0ayTuGQ9ZCwadg== 2023-06-22T18:02:45.613 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQDFjJRkIr3TCBAAyhrdI1X/moUMRrcCeiMuOg== 2023-06-22T18:02:46.386 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQDGjJRkNSfOARAAAyErMCinm7wB+HQovdpllg== 2023-06-22T18:02:46.387 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2023-06-22T18:02:47.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-22T18:02:47.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2023-06-22T18:02:47.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:02:47.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-22T18:02:47.361 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-22T18:02:47.362 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = octopus 2023-06-22T18:02:47.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:02:47.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-22T18:02:47.363 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:02:47.363 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2023-06-22T18:02:51.039 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2023-06-22T18:02:51.274 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-06-22T18:02:51.488 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-06-22T18:02:51.716 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c.target. 2023-06-22T18:02:51.716 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c.target. 2023-06-22T18:02:52.051 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a 2023-06-22T18:02:52.052 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a.service: Unit ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a.service not loaded. 2023-06-22T18:02:52.287 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c.target.wants/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@.service. 2023-06-22T18:02:52.317 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:52 smithi039 systemd[1]: Starting Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:02:53.179 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-22T18:02:53.180 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-06-22T18:02:53.198 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-22T18:02:53.198 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-06-22T18:02:53.198 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-06-22T18:02:53.198 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2023-06-22T18:02:53.199 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2023-06-22T18:02:53.199 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2023-06-22T18:02:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:53 smithi039 bash[108995]: fca454f776e0674ad0b0b27df629908655be1763c8f045d1b52f200357c35a11 2023-06-22T18:02:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:53 smithi039 systemd[1]: Started Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:02:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:53 smithi039 ceph-mon[109059]: mkfs d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:02:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:53 smithi039 ceph-mon[109059]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-22T18:02:54.274 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2023-06-22T18:02:54.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:02:54.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-06-22T18:02:54.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:02:54.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2023-06-22T18:02:54.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.673408s) 2023-06-22T18:02:54.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-06-22T18:02:54.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:02:54.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.745+0000 7f3caee4a700 1 Processor -- start 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.745+0000 7f3caee4a700 1 -- start start 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.745+0000 7f3caee4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.745+0000 7f3caee4a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8076640 con 0x7f3ca8077c90 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.746+0000 7f3cacbe6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.746+0000 7f3cacbe6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41986/0 (socket says 172.21.15.39:41986) 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.746+0000 7f3cacbe6700 1 -- 172.21.15.39:0/463890793 learned_addr learned my addr 172.21.15.39:0/463890793 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:02:54.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.747+0000 7f3cacbe6700 1 -- 172.21.15.39:0/463890793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8076780 con 0x7f3ca8077c90 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.747+0000 7f3cacbe6700 1 --2- 172.21.15.39:0/463890793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8076100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0098e0 tx=0x7f3c9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e73d1ad71c8c251 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.747+0000 7f3ca77fe700 1 -- 172.21.15.39:0/463890793 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c9c009e80 con 0x7f3ca8077c90 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.747+0000 7f3ca77fe700 1 -- 172.21.15.39:0/463890793 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c00b860 con 0x7f3ca8077c90 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.747+0000 7f3ca77fe700 1 -- 172.21.15.39:0/463890793 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c9c00b9d0 con 0x7f3ca8077c90 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 -- 172.21.15.39:0/463890793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 msgr2=0x7f3ca8076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 --2- 172.21.15.39:0/463890793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8076100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0098e0 tx=0x7f3c9c009250 comp rx=0 tx=0).stop 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 -- 172.21.15.39:0/463890793 shutdown_connections 2023-06-22T18:02:54.282 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 --2- 172.21.15.39:0/463890793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8076100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 -- 172.21.15.39:0/463890793 >> 172.21.15.39:0/463890793 conn(0x7f3ca80fd8a0 msgr2=0x7f3ca80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 -- 172.21.15.39:0/463890793 shutdown_connections 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.748+0000 7f3caee4a700 1 -- 172.21.15.39:0/463890793 wait complete. 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.749+0000 7f3caee4a700 1 Processor -- start 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.749+0000 7f3caee4a700 1 -- start start 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.749+0000 7f3caee4a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:02:54.283 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.749+0000 7f3caee4a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8112dc0 con 0x7f3ca8077c90 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3cacbe6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3cacbe6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8112880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41994/0 (socket says 172.21.15.39:41994) 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3cacbe6700 1 -- 172.21.15.39:0/3381089599 learned_addr learned my addr 172.21.15.39:0/3381089599 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3cacbe6700 1 -- 172.21.15.39:0/3381089599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0094f0 con 0x7f3ca8077c90 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3cacbe6700 1 --2- 172.21.15.39:0/3381089599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8112880 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c003080 tx=0x7f3c9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c9c01c460 con 0x7f3ca8077c90 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.750+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c01c5c0 con 0x7f3ca8077c90 2023-06-22T18:02:54.284 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.751+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca8112fc0 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.751+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c9c01c890 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.751+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca81133d0 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.751+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3c9c01a450 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.751+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3c9c02e790 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.752+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca804b970 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.755+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f3c9c01cd80 con 0x7f3ca8077c90 2023-06-22T18:02:54.285 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.831+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3ca810c2b0 con 0x7f3ca8077c90 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.831+0000 7f3ca57fa700 1 -- 172.21.15.39:0/3381089599 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f3c9c01a890 con 0x7f3ca8077c90 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 msgr2=0x7f3ca8112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 --2- 172.21.15.39:0/3381089599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8112880 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c003080 tx=0x7f3c9c016000 comp rx=0 tx=0).stop 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 shutdown_connections 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 --2- 172.21.15.39:0/3381089599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca8077c90 0x7f3ca8112880 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 >> 172.21.15.39:0/3381089599 conn(0x7f3ca80fd8a0 msgr2=0x7f3ca80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 shutdown_connections 2023-06-22T18:02:54.286 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:53.832+0000 7f3caee4a700 1 -- 172.21.15.39:0/3381089599 wait complete. 2023-06-22T18:02:54.287 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2023-06-22T18:02:54.287 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2023-06-22T18:02:54.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:54 smithi039 ceph-mon[109059]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-22T18:02:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:54 smithi039 ceph-mon[109059]: monmap e1: 1 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0]} removed_ranks: {} 2023-06-22T18:02:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:54 smithi039 ceph-mon[109059]: fsmap 2023-06-22T18:02:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:54 smithi039 ceph-mon[109059]: osdmap e1: 0 total, 0 up, 0 in 2023-06-22T18:02:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:54 smithi039 ceph-mon[109059]: mgrmap e1: no daemons active 2023-06-22T18:02:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:54 smithi039 ceph-mon[109059]: from='client.? 172.21.15.39:0/3381089599' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-06-22T18:02:55.298 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:55 smithi039 ceph-mon[109059]: from='client.? 172.21.15.39:0/1928781395' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-22T18:02:55.299 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:02:55 smithi039 ceph-mon[109059]: from='client.? 172.21.15.39:0/1928781395' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-06-22T18:02:55.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:02:55.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2023-06-22T18:02:55.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:02:55.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2023-06-22T18:02:55.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-22T18:02:55.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-22T18:02:55.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2023-06-22T18:02:55.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-06-22T18:02:55.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-06-22T18:02:55.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-06-22T18:02:55.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-22T18:02:55.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-22T18:02:55.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.843+0000 7f530d474700 1 Processor -- start 2023-06-22T18:02:55.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.843+0000 7f530d474700 1 -- start start 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.844+0000 7f530d474700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.844+0000 7f530d474700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5308102b30 con 0x7f53081021e0 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.844+0000 7f5306ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.844+0000 7f5306ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42010/0 (socket says 172.21.15.39:42010) 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.844+0000 7f5306ffd700 1 -- 172.21.15.39:0/3999405378 learned_addr learned my addr 172.21.15.39:0/3999405378 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.844+0000 7f5306ffd700 1 -- 172.21.15.39:0/3999405378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308102c70 con 0x7f53081021e0 2023-06-22T18:02:55.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f5306ffd700 1 --2- 172.21.15.39:0/3999405378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081025f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f52f80098e0 tx=0x7f52f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8768e51ad1911f99 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f5305ffb700 1 -- 172.21.15.39:0/3999405378 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f8009e80 con 0x7f53081021e0 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f5305ffb700 1 -- 172.21.15.39:0/3999405378 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f800b860 con 0x7f53081021e0 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f5305ffb700 1 -- 172.21.15.39:0/3999405378 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f800b9d0 con 0x7f53081021e0 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f530d474700 1 -- 172.21.15.39:0/3999405378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 msgr2=0x7f53081025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f530d474700 1 --2- 172.21.15.39:0/3999405378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081025f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f52f80098e0 tx=0x7f52f8009250 comp rx=0 tx=0).stop 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f530d474700 1 -- 172.21.15.39:0/3999405378 shutdown_connections 2023-06-22T18:02:55.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f530d474700 1 --2- 172.21.15.39:0/3999405378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081025f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:02:55.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.845+0000 7f530d474700 1 -- 172.21.15.39:0/3999405378 >> 172.21.15.39:0/3999405378 conn(0x7f53080fd960 msgr2=0x7f53080ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:02:55.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.846+0000 7f530d474700 1 -- 172.21.15.39:0/3999405378 shutdown_connections 2023-06-22T18:02:55.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.846+0000 7f530d474700 1 -- 172.21.15.39:0/3999405378 wait complete. 2023-06-22T18:02:55.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.846+0000 7f530d474700 1 Processor -- start 2023-06-22T18:02:55.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f530d474700 1 -- start start 2023-06-22T18:02:55.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f530d474700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:02:55.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f530d474700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5308110bf0 con 0x7f53081021e0 2023-06-22T18:02:55.361 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f5306ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:02:55.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f5306ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42012/0 (socket says 172.21.15.39:42012) 2023-06-22T18:02:55.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f5306ffd700 1 -- 172.21.15.39:0/1928781395 learned_addr learned my addr 172.21.15.39:0/1928781395 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:02:55.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.847+0000 7f5306ffd700 1 -- 172.21.15.39:0/1928781395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80094f0 con 0x7f53081021e0 2023-06-22T18:02:55.362 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f5306ffd700 1 --2- 172.21.15.39:0/1928781395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081106b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f52f8003080 tx=0x7f52f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:02:55.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f801c460 con 0x7f53081021e0 2023-06-22T18:02:55.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f801c5c0 con 0x7f53081021e0 2023-06-22T18:02:55.363 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5308110df0 con 0x7f53081021e0 2023-06-22T18:02:55.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52f801c890 con 0x7f53081021e0 2023-06-22T18:02:55.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5308111200 con 0x7f53081021e0 2023-06-22T18:02:55.364 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f52f801a450 con 0x7f53081021e0 2023-06-22T18:02:55.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.848+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f52f802e790 con 0x7f53081021e0 2023-06-22T18:02:55.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.849+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530804b970 con 0x7f53081021e0 2023-06-22T18:02:55.365 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.852+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f52f801cd80 con 0x7f53081021e0 2023-06-22T18:02:55.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.924+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5308109960 con 0x7f53081021e0 2023-06-22T18:02:55.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.929+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f52f801b3e0 con 0x7f53081021e0 2023-06-22T18:02:55.366 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.929+0000 7f52ef7fe700 1 -- 172.21.15.39:0/1928781395 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f52f802e950 con 0x7f53081021e0 2023-06-22T18:02:55.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 msgr2=0x7f53081106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:02:55.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 --2- 172.21.15.39:0/1928781395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081106b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f52f8003080 tx=0x7f52f8016000 comp rx=0 tx=0).stop 2023-06-22T18:02:55.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 shutdown_connections 2023-06-22T18:02:55.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 --2- 172.21.15.39:0/1928781395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53081021e0 0x7f53081106b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:02:55.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 >> 172.21.15.39:0/1928781395 conn(0x7f53080fd960 msgr2=0x7f5308104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:02:55.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 shutdown_connections 2023-06-22T18:02:55.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:02:54.930+0000 7f530d474700 1 -- 172.21.15.39:0/1928781395 wait complete. 2023-06-22T18:02:55.369 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2023-06-22T18:03:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.135+0000 7faf12b6b700 1 Processor -- start 2023-06-22T18:03:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.135+0000 7faf12b6b700 1 -- start start 2023-06-22T18:03:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.135+0000 7faf12b6b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c1025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.135+0000 7faf12b6b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c102bc0 con 0x7faf0c1021e0 2023-06-22T18:03:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.136+0000 7faf10907700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c1025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:02.637 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.136+0000 7faf10907700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c1025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53778/0 (socket says 172.21.15.39:53778) 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.136+0000 7faf10907700 1 -- 172.21.15.39:0/1752789333 learned_addr learned my addr 172.21.15.39:0/1752789333 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.136+0000 7faf10907700 1 -- 172.21.15.39:0/1752789333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c1033f0 con 0x7faf0c1021e0 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.136+0000 7faf10907700 1 --2- 172.21.15.39:0/1752789333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c1025f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faf000098e0 tx=0x7faf00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27475b1f5e41df8c server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf0b7fe700 1 -- 172.21.15.39:0/1752789333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00009e80 con 0x7faf0c1021e0 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf0b7fe700 1 -- 172.21.15.39:0/1752789333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf0000bd10 con 0x7faf0c1021e0 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf0b7fe700 1 -- 172.21.15.39:0/1752789333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00011ce0 con 0x7faf0c1021e0 2023-06-22T18:03:02.638 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 -- 172.21.15.39:0/1752789333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 msgr2=0x7faf0c1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 --2- 172.21.15.39:0/1752789333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c1025f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faf000098e0 tx=0x7faf00009250 comp rx=0 tx=0).stop 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 -- 172.21.15.39:0/1752789333 shutdown_connections 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 --2- 172.21.15.39:0/1752789333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c1025f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 -- 172.21.15.39:0/1752789333 >> 172.21.15.39:0/1752789333 conn(0x7faf0c0fd8a0 msgr2=0x7faf0c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 -- 172.21.15.39:0/1752789333 shutdown_connections 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.137+0000 7faf12b6b700 1 -- 172.21.15.39:0/1752789333 wait complete. 2023-06-22T18:03:02.639 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.138+0000 7faf12b6b700 1 Processor -- start 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.138+0000 7faf12b6b700 1 -- start start 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.139+0000 7faf12b6b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.139+0000 7faf12b6b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c076080 con 0x7faf0c1021e0 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.139+0000 7faf10907700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.139+0000 7faf10907700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53794/0 (socket says 172.21.15.39:53794) 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.139+0000 7faf10907700 1 -- 172.21.15.39:0/2556089108 learned_addr learned my addr 172.21.15.39:0/2556089108 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:02.640 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.139+0000 7faf10907700 1 -- 172.21.15.39:0/2556089108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000094f0 con 0x7faf0c1021e0 2023-06-22T18:03:02.641 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf10907700 1 --2- 172.21.15.39:0/2556089108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c077c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faf00011970 tx=0x7faf00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:02.641 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00009e80 con 0x7faf0c1021e0 2023-06-22T18:03:02.641 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf0001aaf0 con 0x7faf0c1021e0 2023-06-22T18:03:02.641 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c076280 con 0x7faf0c1021e0 2023-06-22T18:03:02.641 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf00022a90 con 0x7faf0c1021e0 2023-06-22T18:03:02.642 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c076690 con 0x7faf0c1021e0 2023-06-22T18:03:02.642 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faf00022c40 con 0x7faf0c1021e0 2023-06-22T18:03:02.642 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.140+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7faf0002b3e0 con 0x7faf0c1021e0 2023-06-22T18:03:02.642 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.141+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0c05f060 con 0x7faf0c1021e0 2023-06-22T18:03:02.642 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.144+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7faf00022400 con 0x7faf0c1021e0 2023-06-22T18:03:02.642 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.216+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7faf0c076a90 con 0x7faf0c1021e0 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.217+0000 7faf097fa700 1 -- 172.21.15.39:0/2556089108 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7faf0001ac60 con 0x7faf0c1021e0 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 msgr2=0x7faf0c077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 --2- 172.21.15.39:0/2556089108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c077c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faf00011970 tx=0x7faf00009250 comp rx=0 tx=0).stop 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 shutdown_connections 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 --2- 172.21.15.39:0/2556089108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0c1021e0 0x7faf0c077c00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 >> 172.21.15.39:0/2556089108 conn(0x7faf0c0fd8a0 msgr2=0x7faf0c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:02.643 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 shutdown_connections 2023-06-22T18:03:02.644 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:02.218+0000 7faf12b6b700 1 -- 172.21.15.39:0/2556089108 wait complete. 2023-06-22T18:03:02.644 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2023-06-22T18:03:02.652 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:02 smithi039 ceph-mon[109059]: from='client.? 172.21.15.39:0/2556089108' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:03:02.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:02 smithi039 systemd[1]: Stopping Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:03:03.378 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:03 smithi039 conmon[109035]: 2023-06-22T18:03:03.114+0000 7f6e72cdb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:03:03.378 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:03 smithi039 conmon[109035]: 2023-06-22T18:03:03.115+0000 7f6e72cdb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-22T18:03:03.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:03 smithi039 bash[109594]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mon-a 2023-06-22T18:03:04.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:03 smithi039 systemd[1]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a.service: Succeeded. 2023-06-22T18:03:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:03 smithi039 systemd[1]: Stopped Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:03:04.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:03 smithi039 systemd[1]: Starting Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:03:04.667 INFO:teuthology.orchestra.run.smithi039.stdout:Setting mon public_network to 172.21.0.0/20 2023-06-22T18:03:04.924 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 bash[109911]: 17d6fb9c7384ed00b70793006bcff32e4ec2f18d74d0ecb128ac931ea22da986 2023-06-22T18:03:04.924 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: set uid:gid to 167:167 (ceph:ceph) 2023-06-22T18:03:04.924 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable), process ceph-mon, pid 8 2023-06-22T18:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: pidfile_write: ignore empty --pid-file 2023-06-22T18:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: load: jerasure load: lrc 2023-06-22T18:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: RocksDB version: 6.15.5 2023-06-22T18:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-06-22T18:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Compile date Jun 21 2023 2023-06-22T18:03:04.925 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: DB SUMMARY 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: DB Session ID: KV0PB6BGGYXZ5BF3E6LI 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: CURRENT file: CURRENT 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: IDENTITY file: IDENTITY 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 97403 ; 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.error_if_exists: 0 2023-06-22T18:03:04.926 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.create_if_missing: 0 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.paranoid_checks: 1 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.env: 0x55ba369c0a80 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.fs: Posix File System 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.info_log: 0x55ba38873da0 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_file_opening_threads: 16 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.statistics: (nil) 2023-06-22T18:03:04.927 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.use_fsync: 0 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_log_file_size: 0 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.keep_log_file_num: 1000 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.recycle_log_file_num: 0 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.allow_fallocate: 1 2023-06-22T18:03:04.928 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.allow_mmap_reads: 0 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.allow_mmap_writes: 0 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.use_direct_reads: 0 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.create_missing_column_families: 0 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.db_log_dir: 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-22T18:03:04.929 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.advise_random_on_open: 1 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.db_write_buffer_size: 0 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.write_buffer_manager: 0x55ba38938930 2023-06-22T18:03:04.930 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.rate_limiter: (nil) 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.wal_recovery_mode: 2 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.enable_thread_tracking: 0 2023-06-22T18:03:04.931 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.enable_pipelined_write: 0 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.unordered_write: 0 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.row_cache: None 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.wal_filter: None 2023-06-22T18:03:04.932 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.allow_ingest_behind: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.preserve_deletes: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.two_write_queues: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.manual_wal_flush: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.atomic_flush: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-22T18:03:04.933 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.log_readahead_size: 0 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.best_efforts_recovery: 0 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-22T18:03:04.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.allow_data_in_errors: 0 2023-06-22T18:03:04.935 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.db_host_id: __hostname__ 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_background_jobs: 2 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_background_compactions: -1 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_subcompactions: 1 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-22T18:03:04.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_total_wal_size: 0 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_open_files: -1 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bytes_per_sync: 0 2023-06-22T18:03:04.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_readahead_size: 0 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_background_flushes: -1 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Compression algorithms supported: 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kZSTD supported: 0 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kXpressCompression supported: 0 2023-06-22T18:03:04.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kLZ4HCCompression supported: 1 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kZlibCompression supported: 1 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kSnappyCompression supported: 1 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kLZ4Compression supported: 1 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: kBZip2Compression supported: 0 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-06-22T18:03:04.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.merge_operator: 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_filter: None 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_filter_factory: None 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.sst_partitioner_factory: None 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-22T18:03:04.940 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ba38842d30) 2023-06-22T18:03:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2023-06-22T18:03:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-22T18:03:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-22T18:03:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2023-06-22T18:03:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2023-06-22T18:03:04.941 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: hash_index_allow_collision: 1 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 1 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x55ba388fb1f0 2023-06-22T18:03:04.942 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2023-06-22T18:03:04.943 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-06-22T18:03:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 4 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.write_buffer_size: 33554432 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_write_buffer_number: 2 2023-06-22T18:03:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression: NoCompression 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression: Disabled 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.prefix_extractor: nullptr 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.num_levels: 7 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-22T18:03:04.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-22T18:03:04.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.level: 32767 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.strategy: 0 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compression_opts.enabled: false 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-22T18:03:04.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.target_file_size_base: 67108864 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-22T18:03:04.949 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-22T18:03:04.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.arena_block_size: 4194304 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.disable_auto_compactions: 0 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-22T18:03:04.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.table_properties_collectors: 2023-06-22T18:03:04.952 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.inplace_update_support: 0 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.bloom_locality: 0 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.max_successive_merges: 0 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-22T18:03:04.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.paranoid_file_checks: 0 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.force_consistency_checks: 1 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.report_bg_io_stats: 0 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.ttl: 2592000 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.enable_blob_files: false 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.min_blob_size: 0 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.blob_file_size: 268435456 2023-06-22T18:03:04.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687456984611720, "job": 1, "event": "recovery_started", "wal_files": [10]} 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2023-06-22T18:03:04.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687456984616519, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92435, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90530, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13289, "raw_average_key_size": 49, "raw_value_size": 83301, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 268, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1687456984, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "dd7a236f-61f6-49fd-b9a0-59c8cbbe5864", "db_session_id": "KV0PB6BGGYXZ5BF3E6LI"}} 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687456984617668, "job": 1, "event": "recovery_finished"} 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55ba3888f180 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: DB pointer 0x55ba388aa000 2023-06-22T18:03:04.956 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-22T18:03:04.957 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 92.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.958 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 92.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.959 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 6.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 6.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:03:04.960 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 92.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.961 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 92.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 22.5 0.00 0.00 1 0.004 0 0 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:03:04.962 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 6.78 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-22T18:03:04.963 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???) e1 preinit fsid d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).mds e1 new map 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).mds e1 print_map 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2023-06-22T18:03:04.964 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:03:04.965 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 systemd[1]: Started Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: monmap e1: 1 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0]} removed_ranks: {} 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: fsmap 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: osdmap e1: 0 total, 0 up, 0 in 2023-06-22T18:03:04.966 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:04 smithi039 ceph-mon[109976]: mgrmap e1: no daemons active 2023-06-22T18:03:05.644 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.198+0000 7f359fec4700 1 Processor -- start 2023-06-22T18:03:05.645 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.198+0000 7f359fec4700 1 -- start start 2023-06-22T18:03:05.645 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.198+0000 7f359fec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:05.645 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.198+0000 7f359fec4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598076640 con 0x7f3598077c90 2023-06-22T18:03:05.645 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.199+0000 7f359dc60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:05.645 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.199+0000 7f359dc60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53796/0 (socket says 172.21.15.39:53796) 2023-06-22T18:03:05.645 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.199+0000 7f359dc60700 1 -- 172.21.15.39:0/173655568 learned_addr learned my addr 172.21.15.39:0/173655568 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.199+0000 7f359dc60700 1 -- 172.21.15.39:0/173655568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598076780 con 0x7f3598077c90 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.199+0000 7f359dc60700 1 --2- 172.21.15.39:0/173655568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598076100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f35940098e0 tx=0x7f3594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ba64d7d37ec920 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359cc5e700 1 -- 172.21.15.39:0/173655568 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3594009e80 con 0x7f3598077c90 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359cc5e700 1 -- 172.21.15.39:0/173655568 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f359400bd10 con 0x7f3598077c90 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359cc5e700 1 -- 172.21.15.39:0/173655568 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3594011ce0 con 0x7f3598077c90 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 -- 172.21.15.39:0/173655568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 msgr2=0x7f3598076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:05.646 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 --2- 172.21.15.39:0/173655568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598076100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f35940098e0 tx=0x7f3594009250 comp rx=0 tx=0).stop 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 -- 172.21.15.39:0/173655568 shutdown_connections 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 --2- 172.21.15.39:0/173655568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598076100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 -- 172.21.15.39:0/173655568 >> 172.21.15.39:0/173655568 conn(0x7f35980fd8a0 msgr2=0x7f35980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 -- 172.21.15.39:0/173655568 shutdown_connections 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.200+0000 7f359fec4700 1 -- 172.21.15.39:0/173655568 wait complete. 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.201+0000 7f359fec4700 1 Processor -- start 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.201+0000 7f359fec4700 1 -- start start 2023-06-22T18:03:05.647 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359fec4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:05.648 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359fec4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981135b0 con 0x7f3598077c90 2023-06-22T18:03:05.648 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359dc60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:05.648 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359dc60700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53804/0 (socket says 172.21.15.39:53804) 2023-06-22T18:03:05.648 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359dc60700 1 -- 172.21.15.39:0/1356296756 learned_addr learned my addr 172.21.15.39:0/1356296756 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:05.648 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359dc60700 1 -- 172.21.15.39:0/1356296756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35940094f0 con 0x7f3598077c90 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.202+0000 7f359dc60700 1 --2- 172.21.15.39:0/1356296756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598113070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3594006b00 tx=0x7f3594009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f359401a640 con 0x7f3598077c90 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f359401ac50 con 0x7f3598077c90 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3594022bb0 con 0x7f3598077c90 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35981137b0 con 0x7f3598077c90 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3598113bc0 con 0x7f3598077c90 2023-06-22T18:03:05.649 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f359402c440 con 0x7f3598077c90 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.203+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f359402b730 con 0x7f3598077c90 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.204+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359804b970 con 0x7f3598077c90 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.207+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f359401adc0 con 0x7f3598077c90 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.279+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f3598108520 con 0x7f3598077c90 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.282+0000 7f358e7fc700 1 -- 172.21.15.39:0/1356296756 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3594022490 con 0x7f3598077c90 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 msgr2=0x7f3598113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 --2- 172.21.15.39:0/1356296756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598113070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3594006b00 tx=0x7f3594009250 comp rx=0 tx=0).stop 2023-06-22T18:03:05.650 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 shutdown_connections 2023-06-22T18:03:05.651 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 --2- 172.21.15.39:0/1356296756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3598077c90 0x7f3598113070 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:05.651 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 >> 172.21.15.39:0/1356296756 conn(0x7f35980fd8a0 msgr2=0x7f35980fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:05.651 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 shutdown_connections 2023-06-22T18:03:05.651 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:05.283+0000 7f359fec4700 1 -- 172.21.15.39:0/1356296756 wait complete. 2023-06-22T18:03:05.652 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2023-06-22T18:03:05.652 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:03:05.652 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2023-06-22T18:03:05.652 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 9283 ... 2023-06-22T18:03:05.949 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x 2023-06-22T18:03:05.950 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x.service: Unit ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x.service not loaded. 2023-06-22T18:03:06.162 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c.target.wants/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@.service. 2023-06-22T18:03:06.286 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:06 smithi039 systemd[1]: Starting Ceph mgr.x for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:03:06.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:06 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1356296756' entity='client.admin' 2023-06-22T18:03:07.057 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-22T18:03:07.058 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-06-22T18:03:07.076 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-22T18:03:07.076 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-06-22T18:03:07.076 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-06-22T18:03:07.076 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2023-06-22T18:03:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-22T18:03:07.094 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-06-22T18:03:07.112 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-22T18:03:07.112 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-06-22T18:03:07.112 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-06-22T18:03:07.113 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2023-06-22T18:03:07.113 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2023-06-22T18:03:07.113 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2023-06-22T18:03:07.384 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:06 smithi039 bash[110350]: 8d605e176040012a6cd05c49562f07b331ffe007e9c8593a9ffe8aa2de7dd497 2023-06-22T18:03:07.384 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:07 smithi039 systemd[1]: Started Ceph mgr.x for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:03:08.127 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:07 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3336391565' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-22T18:03:08.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:08.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:08.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "d572fea0-1126-11ee-9b2b-001a4aab830c", 2023-06-22T18:03:08.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-22T18:03:08.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:08.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-22T18:03:08.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-22T18:03:08.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-22T18:03:08.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:08.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-22T18:03:08.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-22T18:03:08.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-22T18:03:08.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-22T18:03:08.205 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-22T18:03:08.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-22T18:03:08.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-22T18:03:08.206 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-22T18:03:08.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-22T18:03:08.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-22T18:03:08.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-22T18:03:08.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-22T18:03:08.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-22T18:03:08.207 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.208 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:08.209 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:08.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-22T18:03:08.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:08.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-06-22T18:02:53.161053+0000", 2023-06-22T18:03:08.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:08.210 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.670+0000 7f9c248bb700 1 Processor -- start 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.670+0000 7f9c248bb700 1 -- start start 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.671+0000 7f9c248bb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c200760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.671+0000 7f9c248bb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20076600 con 0x7f9c20077c50 2023-06-22T18:03:08.211 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.671+0000 7f9c1e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c200760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.671+0000 7f9c1e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c200760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45548/0 (socket says 172.21.15.39:45548) 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.671+0000 7f9c1e59c700 1 -- 172.21.15.39:0/1431050533 learned_addr learned my addr 172.21.15.39:0/1431050533 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.672+0000 7f9c1e59c700 1 -- 172.21.15.39:0/1431050533 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20076740 con 0x7f9c20077c50 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.672+0000 7f9c1e59c700 1 --2- 172.21.15.39:0/1431050533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c200760c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098e0 tx=0x7f9c10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946c5c895aaf8009 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.672+0000 7f9c1d59a700 1 -- 172.21.15.39:0/1431050533 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c10009e80 con 0x7f9c20077c50 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.672+0000 7f9c1d59a700 1 -- 172.21.15.39:0/1431050533 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c1000bd10 con 0x7f9c20077c50 2023-06-22T18:03:08.212 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.672+0000 7f9c1d59a700 1 -- 172.21.15.39:0/1431050533 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c10011ce0 con 0x7f9c20077c50 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 -- 172.21.15.39:0/1431050533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 msgr2=0x7f9c200760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 --2- 172.21.15.39:0/1431050533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c200760c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098e0 tx=0x7f9c10009250 comp rx=0 tx=0).stop 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 -- 172.21.15.39:0/1431050533 shutdown_connections 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 --2- 172.21.15.39:0/1431050533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c200760c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 -- 172.21.15.39:0/1431050533 >> 172.21.15.39:0/1431050533 conn(0x7f9c200fd8a0 msgr2=0x7f9c200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 -- 172.21.15.39:0/1431050533 shutdown_connections 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.673+0000 7f9c248bb700 1 -- 172.21.15.39:0/1431050533 wait complete. 2023-06-22T18:03:08.213 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.674+0000 7f9c248bb700 1 Processor -- start 2023-06-22T18:03:08.214 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.675+0000 7f9c248bb700 1 -- start start 2023-06-22T18:03:08.214 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.675+0000 7f9c248bb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c20113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:08.214 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.675+0000 7f9c248bb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c201135b0 con 0x7f9c20077c50 2023-06-22T18:03:08.214 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.675+0000 7f9c1e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c20113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:08.214 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.675+0000 7f9c1e59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c20113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45550/0 (socket says 172.21.15.39:45550) 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.675+0000 7f9c1e59c700 1 -- 172.21.15.39:0/3336391565 learned_addr learned my addr 172.21.15.39:0/3336391565 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c1e59c700 1 -- 172.21.15.39:0/3336391565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100094f0 con 0x7f9c20077c50 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c1e59c700 1 --2- 172.21.15.39:0/3336391565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c20113070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9c10006b00 tx=0x7f9c10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c1001a640 con 0x7f9c20077c50 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c201137b0 con 0x7f9c20077c50 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c1001ac50 con 0x7f9c20077c50 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c10022bb0 con 0x7f9c20077c50 2023-06-22T18:03:08.215 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.676+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c20113bc0 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.677+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9c1002c440 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.677+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9c1002b730 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.677+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2004b970 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.682+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f9c1001adc0 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.759+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9c201144b0 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.759+0000 7f9c0effd700 1 -- 172.21.15.39:0/3336391565 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f9c10022490 con 0x7f9c20077c50 2023-06-22T18:03:08.216 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.760+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 msgr2=0x7f9c20113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.760+0000 7f9c248bb700 1 --2- 172.21.15.39:0/3336391565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c20113070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9c10006b00 tx=0x7f9c10009250 comp rx=0 tx=0).stop 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.760+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 shutdown_connections 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.760+0000 7f9c248bb700 1 --2- 172.21.15.39:0/3336391565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c20077c50 0x7f9c20113070 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.760+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 >> 172.21.15.39:0/3336391565 conn(0x7f9c200fd8a0 msgr2=0x7f9c200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.761+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 shutdown_connections 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:07.761+0000 7f9c248bb700 1 -- 172.21.15.39:0/3336391565 wait complete. 2023-06-22T18:03:08.217 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2023-06-22T18:03:08.932 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:08 smithi039 conmon[110391]: 2023-06-22T18:03:08.593+0000 7ff992e721c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T18:03:08.932 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:08 smithi039 conmon[110391]: 2023-06-22T18:03:08.807+0000 7ff992e721c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T18:03:08.933 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:08 smithi039 conmon[110391]: 2023-06-22T18:03:08.927+0000 7ff992e721c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T18:03:09.481 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:09 smithi039 conmon[110391]: 2023-06-22T18:03:09.211+0000 7ff992e721c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:03:09.797 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:09 smithi039 conmon[110391]: 2023-06-22T18:03:09.477+0000 7ff992e721c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T18:03:09.798 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:09 smithi039 conmon[110391]: 2023-06-22T18:03:09.686+0000 7ff992e721c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T18:03:09.798 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:09 smithi039 conmon[110391]: 2023-06-22T18:03:09.793+0000 7ff992e721c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T18:03:10.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:10 smithi039 conmon[110391]: 2023-06-22T18:03:10.627+0000 7ff992e721c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T18:03:11.172 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:10 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/317630006' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-22T18:03:11.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:11.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:11.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "d572fea0-1126-11ee-9b2b-001a4aab830c", 2023-06-22T18:03:11.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-06-22T18:03:11.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-22T18:03:11.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-22T18:03:11.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-06-22T18:03:11.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.183 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-22T18:03:11.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:11.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-22T18:03:11.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-22T18:03:11.184 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-22T18:03:11.189 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-22T18:03:11.190 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-22T18:03:11.190 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-22T18:03:11.190 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.190 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-22T18:03:11.190 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-22T18:03:11.190 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.191 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-22T18:03:11.192 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.193 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-06-22T18:02:53.161053+0000", 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.732+0000 7efdb1ebe700 1 Processor -- start 2023-06-22T18:03:11.194 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.732+0000 7efdb1ebe700 1 -- start start 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.732+0000 7efdb1ebe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac1025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.732+0000 7efdb1ebe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac102bc0 con 0x7efdac1021e0 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.733+0000 7efdab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac1025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.733+0000 7efdab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac1025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45552/0 (socket says 172.21.15.39:45552) 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.733+0000 7efdab7fe700 1 -- 172.21.15.39:0/805331388 learned_addr learned my addr 172.21.15.39:0/805331388 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.733+0000 7efdab7fe700 1 -- 172.21.15.39:0/805331388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac1033f0 con 0x7efdac1021e0 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.733+0000 7efdab7fe700 1 --2- 172.21.15.39:0/805331388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac1025f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0098e0 tx=0x7efd9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f9380535b5bece server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:11.195 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdaa7fc700 1 -- 172.21.15.39:0/805331388 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c009e80 con 0x7efdac1021e0 2023-06-22T18:03:11.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdaa7fc700 1 -- 172.21.15.39:0/805331388 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd9c00bd10 con 0x7efdac1021e0 2023-06-22T18:03:11.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdaa7fc700 1 -- 172.21.15.39:0/805331388 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c011ce0 con 0x7efdac1021e0 2023-06-22T18:03:11.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdb1ebe700 1 -- 172.21.15.39:0/805331388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 msgr2=0x7efdac1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:11.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdb1ebe700 1 --2- 172.21.15.39:0/805331388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac1025f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0098e0 tx=0x7efd9c009250 comp rx=0 tx=0).stop 2023-06-22T18:03:11.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdb1ebe700 1 -- 172.21.15.39:0/805331388 shutdown_connections 2023-06-22T18:03:11.196 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdb1ebe700 1 --2- 172.21.15.39:0/805331388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac1025f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.734+0000 7efdb1ebe700 1 -- 172.21.15.39:0/805331388 >> 172.21.15.39:0/805331388 conn(0x7efdac0fd8a0 msgr2=0x7efdac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.735+0000 7efdb1ebe700 1 -- 172.21.15.39:0/805331388 shutdown_connections 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.735+0000 7efdb1ebe700 1 -- 172.21.15.39:0/805331388 wait complete. 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.735+0000 7efdb1ebe700 1 Processor -- start 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.736+0000 7efdb1ebe700 1 -- start start 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.736+0000 7efdb1ebe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac110ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.736+0000 7efdb1ebe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac111420 con 0x7efdac1021e0 2023-06-22T18:03:11.197 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.736+0000 7efdab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac110ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.736+0000 7efdab7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac110ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45558/0 (socket says 172.21.15.39:45558) 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.736+0000 7efdab7fe700 1 -- 172.21.15.39:0/317630006 learned_addr learned my addr 172.21.15.39:0/317630006 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efdab7fe700 1 -- 172.21.15.39:0/317630006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0094f0 con 0x7efdac1021e0 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efdab7fe700 1 --2- 172.21.15.39:0/317630006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac110ee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd9c011970 tx=0x7efd9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c01a670 con 0x7efdac1021e0 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd9c01ac80 con 0x7efdac1021e0 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac111620 con 0x7efdac1021e0 2023-06-22T18:03:11.198 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd9c022ac0 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.737+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac111a30 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.738+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7efd9c023410 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.738+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efd9c0193c0 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.738+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac05f060 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.741+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7efd9c028060 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.817+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efdac112450 con 0x7efdac1021e0 2023-06-22T18:03:11.199 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.818+0000 7efd93fff700 1 -- 172.21.15.39:0/317630006 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7efd9c01e030 con 0x7efdac1021e0 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 msgr2=0x7efdac110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 --2- 172.21.15.39:0/317630006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac110ee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd9c011970 tx=0x7efd9c009250 comp rx=0 tx=0).stop 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 shutdown_connections 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 --2- 172.21.15.39:0/317630006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdac1021e0 0x7efdac110ee0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 >> 172.21.15.39:0/317630006 conn(0x7efdac0fd8a0 msgr2=0x7efdac0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 shutdown_connections 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:10.819+0000 7efdb1ebe700 1 -- 172.21.15.39:0/317630006 wait complete. 2023-06-22T18:03:11.200 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2023-06-22T18:03:11.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:11 smithi039 conmon[110391]: 2023-06-22T18:03:11.280+0000 7ff992e721c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T18:03:11.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:11 smithi039 conmon[110391]: 2023-06-22T18:03:11.406+0000 7ff992e721c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T18:03:11.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:11 smithi039 conmon[110391]: 2023-06-22T18:03:11.528+0000 7ff992e721c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T18:03:12.669 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:12 smithi039 conmon[110391]: 2023-06-22T18:03:12.320+0000 7ff992e721c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T18:03:12.669 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:12 smithi039 conmon[110391]: 2023-06-22T18:03:12.543+0000 7ff992e721c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:03:12.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:12 smithi039 conmon[110391]: 2023-06-22T18:03:12.664+0000 7ff992e721c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T18:03:12.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:12 smithi039 conmon[110391]: 2023-06-22T18:03:12.840+0000 7ff992e721c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T18:03:13.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:12 smithi039 conmon[110391]: 2023-06-22T18:03:12.963+0000 7ff992e721c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T18:03:14.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:13 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3285688307' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-22T18:03:14.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:14.160 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:14.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "d572fea0-1126-11ee-9b2b-001a4aab830c", 2023-06-22T18:03:14.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-06-22T18:03:14.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-22T18:03:14.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-22T18:03:14.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-22T18:03:14.161 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-22T18:03:14.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-22T18:03:14.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-06-22T18:03:14.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:14.163 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-22T18:03:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-06-22T18:03:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-06-22T18:03:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-22T18:03:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-22T18:03:14.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-22T18:03:14.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-22T18:03:14.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-22T18:03:14.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-22T18:03:14.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-22T18:03:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-22T18:03:14.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-06-22T18:03:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-06-22T18:02:53.161053+0000", 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:14.175 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.722+0000 7f408c74f700 1 Processor -- start 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.723+0000 7f408c74f700 1 -- start start 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.723+0000 7f408c74f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.723+0000 7f408c74f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084076640 con 0x7f4084077c90 2023-06-22T18:03:14.176 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.723+0000 7f408a4eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.723+0000 7f408a4eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45562/0 (socket says 172.21.15.39:45562) 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.723+0000 7f408a4eb700 1 -- 172.21.15.39:0/928010278 learned_addr learned my addr 172.21.15.39:0/928010278 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.724+0000 7f408a4eb700 1 -- 172.21.15.39:0/928010278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084076780 con 0x7f4084077c90 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.724+0000 7f408a4eb700 1 --2- 172.21.15.39:0/928010278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084076100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40800098e0 tx=0x7f4080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800dc94f1201ecbb server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.724+0000 7f40894e9700 1 -- 172.21.15.39:0/928010278 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4080009e80 con 0x7f4084077c90 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.724+0000 7f40894e9700 1 -- 172.21.15.39:0/928010278 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f408000bd10 con 0x7f4084077c90 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.724+0000 7f40894e9700 1 -- 172.21.15.39:0/928010278 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4080011ce0 con 0x7f4084077c90 2023-06-22T18:03:14.177 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 -- 172.21.15.39:0/928010278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 msgr2=0x7f4084076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 --2- 172.21.15.39:0/928010278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084076100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40800098e0 tx=0x7f4080009250 comp rx=0 tx=0).stop 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 -- 172.21.15.39:0/928010278 shutdown_connections 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 --2- 172.21.15.39:0/928010278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084076100 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 -- 172.21.15.39:0/928010278 >> 172.21.15.39:0/928010278 conn(0x7f40840fd8a0 msgr2=0x7f40840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 -- 172.21.15.39:0/928010278 shutdown_connections 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.725+0000 7f408c74f700 1 -- 172.21.15.39:0/928010278 wait complete. 2023-06-22T18:03:14.178 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.726+0000 7f408c74f700 1 Processor -- start 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.726+0000 7f408c74f700 1 -- start start 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.726+0000 7f408c74f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.726+0000 7f408c74f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40841135b0 con 0x7f4084077c90 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.727+0000 7f408a4eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.727+0000 7f408a4eb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45574/0 (socket says 172.21.15.39:45574) 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.727+0000 7f408a4eb700 1 -- 172.21.15.39:0/3285688307 learned_addr learned my addr 172.21.15.39:0/3285688307 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.727+0000 7f408a4eb700 1 -- 172.21.15.39:0/3285688307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800094f0 con 0x7f4084077c90 2023-06-22T18:03:14.179 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.727+0000 7f408a4eb700 1 --2- 172.21.15.39:0/3285688307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084113070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4080006b00 tx=0x7f4080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f408001a640 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f408001ac50 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40841137b0 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4080022bb0 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4084113bc0 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f408002c440 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.728+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f408002b730 con 0x7f4084077c90 2023-06-22T18:03:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.729+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408404b970 con 0x7f4084077c90 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.731+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f408001adc0 con 0x7f4084077c90 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.807+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4084108520 con 0x7f4084077c90 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.808+0000 7f407affd700 1 -- 172.21.15.39:0/3285688307 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f4080022490 con 0x7f4084077c90 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 msgr2=0x7f4084113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 --2- 172.21.15.39:0/3285688307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084113070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4080006b00 tx=0x7f4080009250 comp rx=0 tx=0).stop 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 shutdown_connections 2023-06-22T18:03:14.181 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 --2- 172.21.15.39:0/3285688307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4084077c90 0x7f4084113070 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:14.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 >> 172.21.15.39:0/3285688307 conn(0x7f40840fd8a0 msgr2=0x7f40840fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:14.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 shutdown_connections 2023-06-22T18:03:14.182 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:13.809+0000 7f408c74f700 1 -- 172.21.15.39:0/3285688307 wait complete. 2023-06-22T18:03:14.182 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (3/15)... 2023-06-22T18:03:14.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:14 smithi039 conmon[110391]: 2023-06-22T18:03:14.132+0000 7ff992e721c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T18:03:14.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:14 smithi039 conmon[110391]: 2023-06-22T18:03:14.250+0000 7ff992e721c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T18:03:14.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:14 smithi039 conmon[110391]: 2023-06-22T18:03:14.367+0000 7ff992e721c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T18:03:14.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:14 smithi039 conmon[110391]: 2023-06-22T18:03:14.564+0000 7ff992e721c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T18:03:14.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:14 smithi039 conmon[110391]: 2023-06-22T18:03:14.696+0000 7ff992e721c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T18:03:15.563 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:15 smithi039 conmon[110391]: 2023-06-22T18:03:15.201+0000 7ff992e721c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T18:03:15.563 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:15 smithi039 conmon[110391]: 2023-06-22T18:03:15.321+0000 7ff992e721c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T18:03:15.564 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:15 smithi039 conmon[110391]: 2023-06-22T18:03:15.439+0000 7ff992e721c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T18:03:15.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:15 smithi039 conmon[110391]: 2023-06-22T18:03:15.559+0000 7ff992e721c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T18:03:15.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: Activating manager daemon x 2023-06-22T18:03:15.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: mgrmap e2: x(active, starting, since 0.00599733s) 2023-06-22T18:03:15.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-22T18:03:15.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-22T18:03:15.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-22T18:03:15.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: Manager daemon x is now available 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' 2023-06-22T18:03:15.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:15 smithi039 ceph-mon[109976]: from='mgr.14100 172.21.15.39:0/841614636' entity='mgr.x' 2023-06-22T18:03:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "d572fea0-1126-11ee-9b2b-001a4aab830c", 2023-06-22T18:03:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2023-06-22T18:03:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-22T18:03:17.340 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-22T18:03:17.341 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-22T18:03:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.343 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-22T18:03:17.343 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:17.343 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-22T18:03:17.343 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-22T18:03:17.343 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-22T18:03:17.349 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-22T18:03:17.349 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-22T18:03:17.349 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-22T18:03:17.349 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.349 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-22T18:03:17.350 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2023-06-22T18:03:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2023-06-22T18:02:53.161053+0000", 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-22T18:03:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.699+0000 7f62b4f3b700 1 Processor -- start 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.699+0000 7f62b4f3b700 1 -- start start 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.700+0000 7f62b4f3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b01047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.700+0000 7f62b4f3b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0104d30 con 0x7f62b01043e0 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.700+0000 7f62ae59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b01047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.700+0000 7f62ae59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b01047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45486/0 (socket says 172.21.15.39:45486) 2023-06-22T18:03:17.354 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.700+0000 7f62ae59c700 1 -- 172.21.15.39:0/1676562015 learned_addr learned my addr 172.21.15.39:0/1676562015 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.700+0000 7f62ae59c700 1 -- 172.21.15.39:0/1676562015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b0104e70 con 0x7f62b01043e0 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.701+0000 7f62ae59c700 1 --2- 172.21.15.39:0/1676562015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b01047f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62a00098e0 tx=0x7f62a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd876b48950e2db4 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.701+0000 7f62ad59a700 1 -- 172.21.15.39:0/1676562015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62a0009e80 con 0x7f62b01043e0 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.701+0000 7f62ad59a700 1 -- 172.21.15.39:0/1676562015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62a000bd10 con 0x7f62b01043e0 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.701+0000 7f62ad59a700 1 -- 172.21.15.39:0/1676562015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62a0011ce0 con 0x7f62b01043e0 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 -- 172.21.15.39:0/1676562015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 msgr2=0x7f62b01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 --2- 172.21.15.39:0/1676562015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b01047f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f62a00098e0 tx=0x7f62a0009250 comp rx=0 tx=0).stop 2023-06-22T18:03:17.355 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 -- 172.21.15.39:0/1676562015 shutdown_connections 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 --2- 172.21.15.39:0/1676562015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b01047f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 -- 172.21.15.39:0/1676562015 >> 172.21.15.39:0/1676562015 conn(0x7f62b0077990 msgr2=0x7f62b0102350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 -- 172.21.15.39:0/1676562015 shutdown_connections 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.702+0000 7f62b4f3b700 1 -- 172.21.15.39:0/1676562015 wait complete. 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62b4f3b700 1 Processor -- start 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62b4f3b700 1 -- start start 2023-06-22T18:03:17.356 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62b4f3b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b0113060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62b4f3b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b01135a0 con 0x7f62b01043e0 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62ae59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b0113060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62ae59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b0113060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45498/0 (socket says 172.21.15.39:45498) 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.703+0000 7f62ae59c700 1 -- 172.21.15.39:0/4178046738 learned_addr learned my addr 172.21.15.39:0/4178046738 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f62ae59c700 1 -- 172.21.15.39:0/4178046738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00094f0 con 0x7f62b01043e0 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f62ae59c700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b0113060 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62a0006b00 tx=0x7f62a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62a001a640 con 0x7f62b01043e0 2023-06-22T18:03:17.357 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62a001ac50 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b01137a0 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62a0022bb0 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.704+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b0113bb0 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.705+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 3) v1 ==== 43102+0+0 (secure 0 0 0) 0x7f62a001a7a0 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.705+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b004b970 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.706+0000 7f629effd700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f6298036450 0x7f6298038900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.706+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f62a004a190 con 0x7f62b01043e0 2023-06-22T18:03:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.706+0000 7f62add9b700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f6298036450 0x7f6298038900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.707+0000 7f62add9b700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f6298036450 0x7f6298038900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009990 tx=0x7f62a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.712+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f62a0021580 con 0x7f62b01043e0 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.997+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f62b0114800 con 0x7f62b01043e0 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:16.997+0000 7f629effd700 1 -- 172.21.15.39:0/4178046738 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f62a00213a0 con 0x7f62b01043e0 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f6298036450 msgr2=0x7f6298038900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f6298036450 0x7f6298038900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009990 tx=0x7f62a4006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 msgr2=0x7f62b0113060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:17.359 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b0113060 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62a0006b00 tx=0x7f62a0009250 comp rx=0 tx=0).stop 2023-06-22T18:03:17.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 shutdown_connections 2023-06-22T18:03:17.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f6298036450 0x7f6298038900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:17.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 --2- 172.21.15.39:0/4178046738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62b01043e0 0x7f62b0113060 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:17.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 >> 172.21.15.39:0/4178046738 conn(0x7f62b0077990 msgr2=0x7f62b0101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:17.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 shutdown_connections 2023-06-22T18:03:17.360 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.000+0000 7f62b4f3b700 1 -- 172.21.15.39:0/4178046738 wait complete. 2023-06-22T18:03:17.361 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2023-06-22T18:03:17.686 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:17 smithi039 ceph-mon[109976]: mgrmap e3: x(active, since 1.0104s) 2023-06-22T18:03:17.687 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:17 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/4178046738' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-22T18:03:18.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:18.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2023-06-22T18:03:18.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:03:18.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-22T18:03:18.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-22T18:03:18.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-22T18:03:18.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-22T18:03:18.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:18.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-06-22T18:03:18.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.863+0000 7f7571e55700 1 Processor -- start 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.863+0000 7f7571e55700 1 -- start start 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.863+0000 7f7571e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.863+0000 7f7571e55700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c076640 con 0x7f756c077c90 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.864+0000 7f756b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:18.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.864+0000 7f756b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45512/0 (socket says 172.21.15.39:45512) 2023-06-22T18:03:18.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.864+0000 7f756b7fe700 1 -- 172.21.15.39:0/3268376298 learned_addr learned my addr 172.21.15.39:0/3268376298 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:18.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.864+0000 7f756b7fe700 1 -- 172.21.15.39:0/3268376298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c076780 con 0x7f756c077c90 2023-06-22T18:03:18.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.864+0000 7f756b7fe700 1 --2- 172.21.15.39:0/3268376298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c076100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f755c0098e0 tx=0x7f755c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41854355a1fb8105 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:18.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.864+0000 7f756a7fc700 1 -- 172.21.15.39:0/3268376298 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c009e80 con 0x7f756c077c90 2023-06-22T18:03:18.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f756a7fc700 1 -- 172.21.15.39:0/3268376298 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f755c00bd10 con 0x7f756c077c90 2023-06-22T18:03:18.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f756a7fc700 1 -- 172.21.15.39:0/3268376298 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c011ce0 con 0x7f756c077c90 2023-06-22T18:03:18.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 -- 172.21.15.39:0/3268376298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 msgr2=0x7f756c076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:18.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 --2- 172.21.15.39:0/3268376298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c076100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f755c0098e0 tx=0x7f755c009250 comp rx=0 tx=0).stop 2023-06-22T18:03:18.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 -- 172.21.15.39:0/3268376298 shutdown_connections 2023-06-22T18:03:18.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 --2- 172.21.15.39:0/3268376298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c076100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:18.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 -- 172.21.15.39:0/3268376298 >> 172.21.15.39:0/3268376298 conn(0x7f756c0fd8a0 msgr2=0x7f756c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:18.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 -- 172.21.15.39:0/3268376298 shutdown_connections 2023-06-22T18:03:18.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.865+0000 7f7571e55700 1 -- 172.21.15.39:0/3268376298 wait complete. 2023-06-22T18:03:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.866+0000 7f7571e55700 1 Processor -- start 2023-06-22T18:03:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.866+0000 7f7571e55700 1 -- start start 2023-06-22T18:03:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.866+0000 7f7571e55700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.866+0000 7f7571e55700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c1135b0 con 0x7f756c077c90 2023-06-22T18:03:18.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.867+0000 7f756b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:18.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.867+0000 7f756b7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45516/0 (socket says 172.21.15.39:45516) 2023-06-22T18:03:18.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.867+0000 7f756b7fe700 1 -- 172.21.15.39:0/319645881 learned_addr learned my addr 172.21.15.39:0/319645881 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:18.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.867+0000 7f756b7fe700 1 -- 172.21.15.39:0/319645881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c0094f0 con 0x7f756c077c90 2023-06-22T18:03:18.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.867+0000 7f756b7fe700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c113070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f755c006b00 tx=0x7f755c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:18.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.867+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c01a640 con 0x7f756c077c90 2023-06-22T18:03:18.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.868+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f755c01ac50 con 0x7f756c077c90 2023-06-22T18:03:18.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.868+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c1137b0 con 0x7f756c077c90 2023-06-22T18:03:18.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.868+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c022bb0 con 0x7f756c077c90 2023-06-22T18:03:18.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.868+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c113bc0 con 0x7f756c077c90 2023-06-22T18:03:18.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.869+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 43208+0+0 (secure 0 0 0) 0x7f755c02c440 con 0x7f756c077c90 2023-06-22T18:03:18.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.869+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c04b970 con 0x7f756c077c90 2023-06-22T18:03:18.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.869+0000 7f7553fff700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f7554036560 0x7f7554038a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:18.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.869+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f755c04a520 con 0x7f756c077c90 2023-06-22T18:03:18.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.869+0000 7f756affd700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f7554036560 0x7f7554038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:18.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.870+0000 7f756affd700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f7554036560 0x7f7554038a10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7560009990 tx=0x7f7560006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:18.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:17.874+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f755c021bd0 con 0x7f756c077c90 2023-06-22T18:03:18.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.074+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f756c1146f0 con 0x7f756c077c90 2023-06-22T18:03:18.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.075+0000 7f7553fff700 1 -- 172.21.15.39:0/319645881 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f755c02bda0 con 0x7f756c077c90 2023-06-22T18:03:18.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f7554036560 msgr2=0x7f7554038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:18.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f7554036560 0x7f7554038a10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7560009990 tx=0x7f7560006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:18.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 msgr2=0x7f756c113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:18.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c113070 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f755c006b00 tx=0x7f755c009250 comp rx=0 tx=0).stop 2023-06-22T18:03:18.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 shutdown_connections 2023-06-22T18:03:18.530 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f7554036560 0x7f7554038a10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:18.530 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 --2- 172.21.15.39:0/319645881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f756c077c90 0x7f756c113070 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:18.530 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.077+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 >> 172.21.15.39:0/319645881 conn(0x7f756c0fd8a0 msgr2=0x7f756c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:18.530 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.078+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 shutdown_connections 2023-06-22T18:03:18.531 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:18.078+0000 7f7571e55700 1 -- 172.21.15.39:0/319645881 wait complete. 2023-06-22T18:03:18.531 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2023-06-22T18:03:18.784 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:18 smithi039 ceph-mon[109976]: mgrmap e4: x(active, since 2s) 2023-06-22T18:03:18.785 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:18 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/319645881' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-22T18:03:19.924 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:19 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3800246711' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-06-22T18:03:19.925 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:19 smithi039 conmon[110391]: ignoring --setuser ceph since I am not root 2023-06-22T18:03:19.925 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:19 smithi039 conmon[110391]: ignoring --setgroup ceph since I am not root 2023-06-22T18:03:19.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.040+0000 7ff3cc8e9700 1 Processor -- start 2023-06-22T18:03:19.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.041+0000 7ff3cc8e9700 1 -- start start 2023-06-22T18:03:19.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.041+0000 7ff3cc8e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c81025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:19.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.041+0000 7ff3cc8e9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8102b30 con 0x7ff3c81021e0 2023-06-22T18:03:19.990 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.041+0000 7ff3c659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c81025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.041+0000 7ff3c659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c81025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45524/0 (socket says 172.21.15.39:45524) 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.041+0000 7ff3c659c700 1 -- 172.21.15.39:0/2355336692 learned_addr learned my addr 172.21.15.39:0/2355336692 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.042+0000 7ff3c659c700 1 -- 172.21.15.39:0/2355336692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c8102c70 con 0x7ff3c81021e0 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.042+0000 7ff3c659c700 1 --2- 172.21.15.39:0/2355336692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c81025f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80098e0 tx=0x7ff3b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bad391b044ca707 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.042+0000 7ff3c559a700 1 -- 172.21.15.39:0/2355336692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b8009e80 con 0x7ff3c81021e0 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.042+0000 7ff3c559a700 1 -- 172.21.15.39:0/2355336692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3b800bd10 con 0x7ff3c81021e0 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.042+0000 7ff3c559a700 1 -- 172.21.15.39:0/2355336692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b8011ce0 con 0x7ff3c81021e0 2023-06-22T18:03:19.991 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/2355336692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 msgr2=0x7ff3c81025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 --2- 172.21.15.39:0/2355336692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c81025f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80098e0 tx=0x7ff3b8009250 comp rx=0 tx=0).stop 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/2355336692 shutdown_connections 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 --2- 172.21.15.39:0/2355336692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c81025f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/2355336692 >> 172.21.15.39:0/2355336692 conn(0x7ff3c80fd960 msgr2=0x7ff3c80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/2355336692 shutdown_connections 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.043+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/2355336692 wait complete. 2023-06-22T18:03:19.992 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.044+0000 7ff3cc8e9700 1 Processor -- start 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.044+0000 7ff3cc8e9700 1 -- start start 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.044+0000 7ff3cc8e9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c8113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.044+0000 7ff3cc8e9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c81135c0 con 0x7ff3c81021e0 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.044+0000 7ff3c659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c8113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3c659c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c8113080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45532/0 (socket says 172.21.15.39:45532) 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3c659c700 1 -- 172.21.15.39:0/3800246711 learned_addr learned my addr 172.21.15.39:0/3800246711 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3c659c700 1 -- 172.21.15.39:0/3800246711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80094f0 con 0x7ff3c81021e0 2023-06-22T18:03:19.993 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3c659c700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c8113080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8006b00 tx=0x7ff3b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b801a640 con 0x7ff3c81021e0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3b801ac50 con 0x7ff3c81021e0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3b8022bb0 con 0x7ff3c81021e0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.045+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c81137c0 con 0x7ff3c81021e0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.046+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c8113bd0 con 0x7ff3c81021e0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.047+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) v1 ==== 43208+0+0 (secure 0 0 0) 0x7ff3b8022490 con 0x7ff3c81021e0 2023-06-22T18:03:19.994 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.047+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c804b970 con 0x7ff3c81021e0 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.047+0000 7ff3b6ffd700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7ff3b0036560 0x7ff3b0038a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.047+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff3b804a320 con 0x7ff3c81021e0 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.047+0000 7ff3c5d9b700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7ff3b0036560 0x7ff3b0038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.048+0000 7ff3c5d9b700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7ff3b0036560 0x7ff3b0038a10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009990 tx=0x7ff3bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.052+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff3b801adc0 con 0x7ff3c81021e0 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.308+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7ff3c8114820 con 0x7ff3c81021e0 2023-06-22T18:03:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.587+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 5) v1 ==== 43219+0+0 (secure 0 0 0) 0x7ff3b802c960 con 0x7ff3c81021e0 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.589+0000 7ff3b6ffd700 1 -- 172.21.15.39:0/3800246711 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7ff3b804a580 con 0x7ff3c81021e0 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7ff3b0036560 msgr2=0x7ff3b0038a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7ff3b0036560 0x7ff3b0038a10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009990 tx=0x7ff3bc006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 msgr2=0x7ff3c8113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c8113080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8006b00 tx=0x7ff3b8009250 comp rx=0 tx=0).stop 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 shutdown_connections 2023-06-22T18:03:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7ff3b0036560 0x7ff3b0038a10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.591+0000 7ff3cc8e9700 1 --2- 172.21.15.39:0/3800246711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3c81021e0 0x7ff3c8113080 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.592+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 >> 172.21.15.39:0/3800246711 conn(0x7ff3c80fd960 msgr2=0x7ff3c80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.592+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 shutdown_connections 2023-06-22T18:03:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:19.592+0000 7ff3cc8e9700 1 -- 172.21.15.39:0/3800246711 wait complete. 2023-06-22T18:03:20.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:20 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3800246711' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-06-22T18:03:20.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:20 smithi039 ceph-mon[109976]: mgrmap e5: x(active, since 4s) 2023-06-22T18:03:21.165 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.506+0000 7f300da7c700 1 Processor -- start 2023-06-22T18:03:21.166 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.506+0000 7f300da7c700 1 -- start start 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.507+0000 7f300da7c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f30081025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.507+0000 7f300da7c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008102bc0 con 0x7f30081021e0 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.507+0000 7f30077fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f30081025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.507+0000 7f30077fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f30081025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45560/0 (socket says 172.21.15.39:45560) 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.507+0000 7f30077fe700 1 -- 172.21.15.39:0/3270479692 learned_addr learned my addr 172.21.15.39:0/3270479692 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.507+0000 7f30077fe700 1 -- 172.21.15.39:0/3270479692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30081033f0 con 0x7f30081021e0 2023-06-22T18:03:21.167 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.508+0000 7f30077fe700 1 --2- 172.21.15.39:0/3270479692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f30081025f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80098e0 tx=0x7f2ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9262110e7f3fc9c server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.508+0000 7f30067fc700 1 -- 172.21.15.39:0/3270479692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8009e80 con 0x7f30081021e0 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.508+0000 7f30067fc700 1 -- 172.21.15.39:0/3270479692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ff800bd10 con 0x7f30081021e0 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.508+0000 7f30067fc700 1 -- 172.21.15.39:0/3270479692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8011ce0 con 0x7f30081021e0 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.508+0000 7f300da7c700 1 -- 172.21.15.39:0/3270479692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 msgr2=0x7f30081025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.508+0000 7f300da7c700 1 --2- 172.21.15.39:0/3270479692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f30081025f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80098e0 tx=0x7f2ff8009250 comp rx=0 tx=0).stop 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.509+0000 7f300da7c700 1 -- 172.21.15.39:0/3270479692 shutdown_connections 2023-06-22T18:03:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.509+0000 7f300da7c700 1 --2- 172.21.15.39:0/3270479692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f30081025f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.509+0000 7f300da7c700 1 -- 172.21.15.39:0/3270479692 >> 172.21.15.39:0/3270479692 conn(0x7f30080fd8a0 msgr2=0x7f30080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.509+0000 7f300da7c700 1 -- 172.21.15.39:0/3270479692 shutdown_connections 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.509+0000 7f300da7c700 1 -- 172.21.15.39:0/3270479692 wait complete. 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f300da7c700 1 Processor -- start 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f300da7c700 1 -- start start 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f300da7c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f3008113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:21.169 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f300da7c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008113640 con 0x7f30081021e0 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f30077fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f3008113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f30077fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f3008113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45564/0 (socket says 172.21.15.39:45564) 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.510+0000 7f30077fe700 1 -- 172.21.15.39:0/2277837508 learned_addr learned my addr 172.21.15.39:0/2277837508 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.511+0000 7f30077fe700 1 -- 172.21.15.39:0/2277837508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80094f0 con 0x7f30081021e0 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.511+0000 7f30077fe700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f3008113100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8011970 tx=0x7f2ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.511+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8009e80 con 0x7f30081021e0 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.511+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ff801aaf0 con 0x7f30081021e0 2023-06-22T18:03:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.511+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3008113840 con 0x7f30081021e0 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.512+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8022a90 con 0x7f30081021e0 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.512+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3008113c50 con 0x7f30081021e0 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.512+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f2ff8022cc0 con 0x7f30081021e0 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.513+0000 7f2feffff700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 0x7f2ff00386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.513+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300805f060 con 0x7f30081021e0 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.513+0000 7f3006ffd700 1 -- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 msgr2=0x7f2ff00386e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:21.171 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.513+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2ff804bbf0 con 0x7f30081021e0 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.513+0000 7f3006ffd700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 0x7f2ff00386e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.518+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f2ff8050060 con 0x7f30081021e0 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.714+0000 7f3006ffd700 1 -- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 msgr2=0x7f2ff00386e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.714+0000 7f3006ffd700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 0x7f2ff00386e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.775+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3008114770 con 0x7f30081021e0 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.776+0000 7f2feffff700 1 -- 172.21.15.39:0/2277837508 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f2ff8050070 con 0x7f30081021e0 2023-06-22T18:03:21.172 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.778+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 msgr2=0x7f2ff00386e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.778+0000 7f300da7c700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 0x7f2ff00386e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.778+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 msgr2=0x7f3008113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.778+0000 7f300da7c700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f3008113100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8011970 tx=0x7f2ff8009250 comp rx=0 tx=0).stop 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.779+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 shutdown_connections 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.779+0000 7f300da7c700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f2ff0036230 0x7f2ff00386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.779+0000 7f300da7c700 1 --2- 172.21.15.39:0/2277837508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30081021e0 0x7f3008113100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:21.173 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.779+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 >> 172.21.15.39:0/2277837508 conn(0x7f30080fd8a0 msgr2=0x7f30080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:21.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.779+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 shutdown_connections 2023-06-22T18:03:21.174 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:20.779+0000 7f300da7c700 1 -- 172.21.15.39:0/2277837508 wait complete. 2023-06-22T18:03:21.174 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2023-06-22T18:03:21.174 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2023-06-22T18:03:21.592 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:21 smithi039 conmon[110391]: 2023-06-22T18:03:21.278+0000 7fc9bf2bc1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T18:03:21.593 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:21 smithi039 conmon[110391]: 2023-06-22T18:03:21.508+0000 7fc9bf2bc1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T18:03:21.916 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:21 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2277837508' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-22T18:03:21.916 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:21 smithi039 conmon[110391]: 2023-06-22T18:03:21.634+0000 7fc9bf2bc1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T18:03:22.178 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:21 smithi039 conmon[110391]: 2023-06-22T18:03:21.911+0000 7fc9bf2bc1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:03:22.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:22 smithi039 conmon[110391]: 2023-06-22T18:03:22.174+0000 7fc9bf2bc1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T18:03:22.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:22 smithi039 conmon[110391]: 2023-06-22T18:03:22.373+0000 7fc9bf2bc1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T18:03:22.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:22 smithi039 conmon[110391]: 2023-06-22T18:03:22.481+0000 7fc9bf2bc1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T18:03:23.694 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:23 smithi039 conmon[110391]: 2023-06-22T18:03:23.265+0000 7fc9bf2bc1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T18:03:24.194 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:23 smithi039 conmon[110391]: 2023-06-22T18:03:23.886+0000 7fc9bf2bc1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T18:03:24.195 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:24 smithi039 conmon[110391]: 2023-06-22T18:03:24.008+0000 7fc9bf2bc1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T18:03:24.195 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:24 smithi039 conmon[110391]: 2023-06-22T18:03:24.130+0000 7fc9bf2bc1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T18:03:25.194 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:24 smithi039 conmon[110391]: 2023-06-22T18:03:24.922+0000 7fc9bf2bc1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T18:03:25.195 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:25 smithi039 conmon[110391]: 2023-06-22T18:03:25.147+0000 7fc9bf2bc1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:03:25.571 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:25 smithi039 conmon[110391]: 2023-06-22T18:03:25.268+0000 7fc9bf2bc1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T18:03:25.571 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:25 smithi039 conmon[110391]: 2023-06-22T18:03:25.443+0000 7fc9bf2bc1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T18:03:26.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:25 smithi039 conmon[110391]: 2023-06-22T18:03:25.566+0000 7fc9bf2bc1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T18:03:26.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:26 smithi039 conmon[110391]: 2023-06-22T18:03:26.687+0000 7fc9bf2bc1c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T18:03:26.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:26 smithi039 conmon[110391]: 2023-06-22T18:03:26.804+0000 7fc9bf2bc1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T18:03:26.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:26 smithi039 conmon[110391]: 2023-06-22T18:03:26.923+0000 7fc9bf2bc1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T18:03:27.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:27 smithi039 conmon[110391]: 2023-06-22T18:03:27.120+0000 7fc9bf2bc1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T18:03:27.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:27 smithi039 conmon[110391]: 2023-06-22T18:03:27.251+0000 7fc9bf2bc1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T18:03:28.100 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:27 smithi039 conmon[110391]: 2023-06-22T18:03:27.738+0000 7fc9bf2bc1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T18:03:28.101 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:27 smithi039 conmon[110391]: 2023-06-22T18:03:27.859+0000 7fc9bf2bc1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T18:03:28.101 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:27 smithi039 conmon[110391]: 2023-06-22T18:03:27.978+0000 7fc9bf2bc1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T18:03:28.101 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:28 smithi039 conmon[110391]: 2023-06-22T18:03:28.097+0000 7fc9bf2bc1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T18:03:28.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: Active manager daemon x restarted 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: Activating manager daemon x 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: osdmap e2: 0 total, 0 up, 0 in 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: mgrmap e6: x(active, starting, since 0.00707979s) 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-22T18:03:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-22T18:03:28.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-22T18:03:28.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: Manager daemon x is now available 2023-06-22T18:03:28.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:28.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:28.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:28.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:28 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:29 smithi039 ceph-mon[109976]: Found migration_current of "None". Setting to last migration. 2023-06-22T18:03:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:29 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:29 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-22T18:03:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:29 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-22T18:03:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:29 smithi039 ceph-mon[109976]: mgrmap e7: x(active, since 1.00948s) 2023-06-22T18:03:29.549 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:29.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-06-22T18:03:29.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-22T18:03:29.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:29.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.671+0000 7f664ada1700 1 Processor -- start 2023-06-22T18:03:29.550 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.671+0000 7f664ada1700 1 -- start start 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.671+0000 7f664ada1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f66440760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.671+0000 7f664ada1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6644076620 con 0x7f6644077c70 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.672+0000 7f6648b3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f66440760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.672+0000 7f6648b3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f66440760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45576/0 (socket says 172.21.15.39:45576) 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.672+0000 7f6648b3d700 1 -- 172.21.15.39:0/274700744 learned_addr learned my addr 172.21.15.39:0/274700744 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.672+0000 7f6648b3d700 1 -- 172.21.15.39:0/274700744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644076760 con 0x7f6644077c70 2023-06-22T18:03:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.672+0000 7f6648b3d700 1 --2- 172.21.15.39:0/274700744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f66440760e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f66380098e0 tx=0x7f6638009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8bd1e102d33922 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.673+0000 7f66437fe700 1 -- 172.21.15.39:0/274700744 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6638009e80 con 0x7f6644077c70 2023-06-22T18:03:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.673+0000 7f66437fe700 1 -- 172.21.15.39:0/274700744 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f663800bd10 con 0x7f6644077c70 2023-06-22T18:03:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.673+0000 7f66437fe700 1 -- 172.21.15.39:0/274700744 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6638011ce0 con 0x7f6644077c70 2023-06-22T18:03:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.673+0000 7f664ada1700 1 -- 172.21.15.39:0/274700744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 msgr2=0x7f66440760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.673+0000 7f664ada1700 1 --2- 172.21.15.39:0/274700744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f66440760e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f66380098e0 tx=0x7f6638009250 comp rx=0 tx=0).stop 2023-06-22T18:03:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.674+0000 7f664ada1700 1 -- 172.21.15.39:0/274700744 shutdown_connections 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.674+0000 7f664ada1700 1 --2- 172.21.15.39:0/274700744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f66440760e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.674+0000 7f664ada1700 1 -- 172.21.15.39:0/274700744 >> 172.21.15.39:0/274700744 conn(0x7f66440fd8a0 msgr2=0x7f66440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.674+0000 7f664ada1700 1 -- 172.21.15.39:0/274700744 shutdown_connections 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.674+0000 7f664ada1700 1 -- 172.21.15.39:0/274700744 wait complete. 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.674+0000 7f664ada1700 1 Processor -- start 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.675+0000 7f664ada1700 1 -- start start 2023-06-22T18:03:29.553 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.675+0000 7f664ada1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f6644113060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.675+0000 7f664ada1700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66441135a0 con 0x7f6644077c70 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.675+0000 7f6648b3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f6644113060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.675+0000 7f6648b3d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f6644113060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45588/0 (socket says 172.21.15.39:45588) 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.675+0000 7f6648b3d700 1 -- 172.21.15.39:0/3053141972 learned_addr learned my addr 172.21.15.39:0/3053141972 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f6648b3d700 1 -- 172.21.15.39:0/3053141972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380094f0 con 0x7f6644077c70 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f6648b3d700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f6644113060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6638006b00 tx=0x7f6638009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:29.554 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f663801a640 con 0x7f6644077c70 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f663801ac50 con 0x7f6644077c70 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6638022bb0 con 0x7f6644077c70 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66441137a0 con 0x7f6644077c70 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.676+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6644113bb0 con 0x7f6644077c70 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.677+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f663802c440 con 0x7f6644077c70 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.678+0000 7f66417fa700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:29.555 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.678+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 --> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6634039180 con 0x7f66340365c0 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.678+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f663804a3f0 con 0x7f6644077c70 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.678+0000 7f6643fff700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.678+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.879+0000 7f6643fff700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:21.879+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:22.280+0000 7f6643fff700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:29.556 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:22.280+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:23.081+0000 7f6643fff700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:23.081+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:24.682+0000 7f6643fff700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:24.682+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:27.886+0000 7f6643fff700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/2590139173 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:27.886+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-06-22T18:03:29.557 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:28.105+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 6) v1 ==== 42986+0+0 (secure 0 0 0) 0x7f663802b3e0 con 0x7f6644077c70 2023-06-22T18:03:29.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:28.105+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 msgr2=0x7f6634038a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:03:29.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:28.105+0000 7f66417fa700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:29.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.106+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 7) v1 ==== 43113+0+0 (secure 0 0 0) 0x7f6638022d10 con 0x7f6644077c70 2023-06-22T18:03:29.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.106+0000 7f66417fa700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f6634039e90 0x7f663403c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:29.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.106+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6634039180 con 0x7f6634039e90 2023-06-22T18:03:29.558 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.107+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f6634039e90 0x7f663403c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.109+0000 7f6643fff700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f6634039e90 0x7f663403c270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f66300039d0 tx=0x7f6630009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.110+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f6634039180 con 0x7f6634039e90 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.116+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f664404b970 con 0x7f6634039e90 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.116+0000 7f66417fa700 1 -- 172.21.15.39:0/3053141972 <== mgr.14120 v2:172.21.15.39:6800/349518747 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f664404b970 con 0x7f6634039e90 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.116+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f6634039e90 msgr2=0x7f663403c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.116+0000 7f664ada1700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f6634039e90 0x7f663403c270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f66300039d0 tx=0x7f6630009350 comp rx=0 tx=0).stop 2023-06-22T18:03:29.559 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 msgr2=0x7f6644113060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:29.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f6644113060 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6638006b00 tx=0x7f6638009250 comp rx=0 tx=0).stop 2023-06-22T18:03:29.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 shutdown_connections 2023-06-22T18:03:29.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f6634039e90 0x7f663403c270 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:29.560 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6644077c70 0x7f6644113060 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 --2- 172.21.15.39:0/3053141972 >> [v2:172.21.15.39:6800/2590139173,v1:172.21.15.39:6801/2590139173] conn(0x7f66340365c0 0x7f6634038a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 >> 172.21.15.39:0/3053141972 conn(0x7f66440fd8a0 msgr2=0x7f66440fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 shutdown_connections 2023-06-22T18:03:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:29.117+0000 7f664ada1700 1 -- 172.21.15.39:0/3053141972 wait complete. 2023-06-22T18:03:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2023-06-22T18:03:29.561 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2023-06-22T18:03:30.726 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.073+0000 7f996f493700 1 Processor -- start 2023-06-22T18:03:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.073+0000 7f996f493700 1 -- start start 2023-06-22T18:03:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.074+0000 7f996f493700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f99680760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.074+0000 7f996f493700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9968076600 con 0x7f9968077c50 2023-06-22T18:03:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.074+0000 7f996d22f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f99680760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.074+0000 7f996d22f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f99680760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38542/0 (socket says 172.21.15.39:38542) 2023-06-22T18:03:30.727 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.074+0000 7f996d22f700 1 -- 172.21.15.39:0/2961200902 learned_addr learned my addr 172.21.15.39:0/2961200902 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.074+0000 7f996d22f700 1 -- 172.21.15.39:0/2961200902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968076740 con 0x7f9968077c50 2023-06-22T18:03:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.075+0000 7f996d22f700 1 --2- 172.21.15.39:0/2961200902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f99680760c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99640098e0 tx=0x7f9964009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7025c98bc142a34 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.075+0000 7f995ffff700 1 -- 172.21.15.39:0/2961200902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9964009e80 con 0x7f9968077c50 2023-06-22T18:03:30.728 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.075+0000 7f995ffff700 1 -- 172.21.15.39:0/2961200902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f996400bd10 con 0x7f9968077c50 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.075+0000 7f995ffff700 1 -- 172.21.15.39:0/2961200902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9964011ce0 con 0x7f9968077c50 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 -- 172.21.15.39:0/2961200902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 msgr2=0x7f99680760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 --2- 172.21.15.39:0/2961200902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f99680760c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99640098e0 tx=0x7f9964009250 comp rx=0 tx=0).stop 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 -- 172.21.15.39:0/2961200902 shutdown_connections 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 --2- 172.21.15.39:0/2961200902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f99680760c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 -- 172.21.15.39:0/2961200902 >> 172.21.15.39:0/2961200902 conn(0x7f99680fd8a0 msgr2=0x7f99680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:30.729 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 -- 172.21.15.39:0/2961200902 shutdown_connections 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.076+0000 7f996f493700 1 -- 172.21.15.39:0/2961200902 wait complete. 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996f493700 1 Processor -- start 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996f493700 1 -- start start 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996f493700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f9968113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996f493700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99681135b0 con 0x7f9968077c50 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996d22f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f9968113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996d22f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f9968113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38552/0 (socket says 172.21.15.39:38552) 2023-06-22T18:03:30.730 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.077+0000 7f996d22f700 1 -- 172.21.15.39:0/2423912841 learned_addr learned my addr 172.21.15.39:0/2423912841 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f996d22f700 1 -- 172.21.15.39:0/2423912841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99640094f0 con 0x7f9968077c50 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f996d22f700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f9968113070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9964006b00 tx=0x7f9964009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f996401a640 con 0x7f9968077c50 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f996401ac50 con 0x7f9968077c50 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99681137b0 con 0x7f9968077c50 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9964022bb0 con 0x7f9968077c50 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.078+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9968113bc0 con 0x7f9968077c50 2023-06-22T18:03:30.731 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.079+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996804b970 con 0x7f9968077c50 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.079+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 7) v1 ==== 43113+0+0 (secure 0 0 0) 0x7f996401a7a0 con 0x7f9968077c50 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.080+0000 7f995dffb700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f99540364a0 0x7f9954038950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.080+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f996404a270 con 0x7f9968077c50 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.080+0000 7f996ca2e700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f99540364a0 0x7f9954038950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.081+0000 7f996ca2e700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f99540364a0 0x7f9954038950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9958009990 tx=0x7f9958006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.085+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f99640219f0 con 0x7f9968077c50 2023-06-22T18:03:30.732 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.304+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f99680fed20 con 0x7f99540364a0 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.313+0000 7f995dffb700 1 -- 172.21.15.39:0/2423912841 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f99680fed20 con 0x7f99540364a0 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.315+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f99540364a0 msgr2=0x7f9954038950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.315+0000 7f996f493700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f99540364a0 0x7f9954038950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9958009990 tx=0x7f9958006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.315+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 msgr2=0x7f9968113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.315+0000 7f996f493700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f9968113070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9964006b00 tx=0x7f9964009250 comp rx=0 tx=0).stop 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.316+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 shutdown_connections 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.316+0000 7f996f493700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f99540364a0 0x7f9954038950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:30.733 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.316+0000 7f996f493700 1 --2- 172.21.15.39:0/2423912841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9968077c50 0x7f9968113070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.316+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 >> 172.21.15.39:0/2423912841 conn(0x7f99680fd8a0 msgr2=0x7f99680fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.316+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 shutdown_connections 2023-06-22T18:03:30.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:30.316+0000 7f996f493700 1 -- 172.21.15.39:0/2423912841 wait complete. 2023-06-22T18:03:30.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:30 smithi039 ceph-mon[109976]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-22T18:03:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:30 smithi039 ceph-mon[109976]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-22T18:03:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:30 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:30 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:31.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:31 smithi039 ceph-mon[109976]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:31.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:31 smithi039 ceph-mon[109976]: mgrmap e8: x(active, since 2s) 2023-06-22T18:03:31.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:31 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:31.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:31 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:31.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2023-06-22T18:03:31.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.229+0000 7ff05aea7700 1 Processor -- start 2023-06-22T18:03:31.900 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.230+0000 7ff05aea7700 1 -- start start 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.230+0000 7ff05aea7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff0541025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.230+0000 7ff05aea7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054102b30 con 0x7ff0541021e0 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.230+0000 7ff058c43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff0541025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.230+0000 7ff058c43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff0541025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38564/0 (socket says 172.21.15.39:38564) 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.230+0000 7ff058c43700 1 -- 172.21.15.39:0/3069759182 learned_addr learned my addr 172.21.15.39:0/3069759182 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.231+0000 7ff058c43700 1 -- 172.21.15.39:0/3069759182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054102c70 con 0x7ff0541021e0 2023-06-22T18:03:31.901 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.231+0000 7ff058c43700 1 --2- 172.21.15.39:0/3069759182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff0541025f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff0480098e0 tx=0x7ff048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d56d349495cec8 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.231+0000 7ff0537fe700 1 -- 172.21.15.39:0/3069759182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff048009e80 con 0x7ff0541021e0 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.231+0000 7ff0537fe700 1 -- 172.21.15.39:0/3069759182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff04800bd10 con 0x7ff0541021e0 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.231+0000 7ff0537fe700 1 -- 172.21.15.39:0/3069759182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff048011ce0 con 0x7ff0541021e0 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 -- 172.21.15.39:0/3069759182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 msgr2=0x7ff0541025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 --2- 172.21.15.39:0/3069759182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff0541025f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff0480098e0 tx=0x7ff048009250 comp rx=0 tx=0).stop 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 -- 172.21.15.39:0/3069759182 shutdown_connections 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 --2- 172.21.15.39:0/3069759182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff0541025f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:31.902 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 -- 172.21.15.39:0/3069759182 >> 172.21.15.39:0/3069759182 conn(0x7ff0540fd960 msgr2=0x7ff0540ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 -- 172.21.15.39:0/3069759182 shutdown_connections 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.232+0000 7ff05aea7700 1 -- 172.21.15.39:0/3069759182 wait complete. 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.233+0000 7ff05aea7700 1 Processor -- start 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.233+0000 7ff05aea7700 1 -- start start 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.233+0000 7ff05aea7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff054110ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.233+0000 7ff05aea7700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054111400 con 0x7ff0541021e0 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff058c43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff054110ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:31.903 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff058c43700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff054110ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38576/0 (socket says 172.21.15.39:38576) 2023-06-22T18:03:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff058c43700 1 -- 172.21.15.39:0/3097083345 learned_addr learned my addr 172.21.15.39:0/3097083345 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff058c43700 1 -- 172.21.15.39:0/3097083345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480094f0 con 0x7ff0541021e0 2023-06-22T18:03:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff058c43700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff054110ec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff048006b00 tx=0x7ff048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff04801a640 con 0x7ff0541021e0 2023-06-22T18:03:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff04801ac50 con 0x7ff0541021e0 2023-06-22T18:03:31.904 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff048022bb0 con 0x7ff0541021e0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.234+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff054111600 con 0x7ff0541021e0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.235+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0541121c0 con 0x7ff0541021e0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.235+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7ff048022490 con 0x7ff0541021e0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.235+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05405f060 con 0x7ff0541021e0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.236+0000 7ff0517fa700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff0440365c0 0x7ff044038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.236+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff04804a480 con 0x7ff0541021e0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.236+0000 7ff053fff700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff0440365c0 0x7ff044038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:31.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.237+0000 7ff053fff700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff0440365c0 0x7ff044038a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff040009990 tx=0x7ff040006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.241+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff04801adc0 con 0x7ff0541021e0 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.449+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0540fed70 con 0x7ff0440365c0 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.450+0000 7ff0517fa700 1 -- 172.21.15.39:0/3097083345 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ff0540fed70 con 0x7ff0440365c0 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff0440365c0 msgr2=0x7ff044038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff0440365c0 0x7ff044038a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff040009990 tx=0x7ff040006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 msgr2=0x7ff054110ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff054110ec0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff048006b00 tx=0x7ff048009250 comp rx=0 tx=0).stop 2023-06-22T18:03:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 shutdown_connections 2023-06-22T18:03:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff0440365c0 0x7ff044038a70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 --2- 172.21.15.39:0/3097083345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0541021e0 0x7ff054110ec0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 >> 172.21.15.39:0/3097083345 conn(0x7ff0540fd960 msgr2=0x7ff0540fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 shutdown_connections 2023-06-22T18:03:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:31.452+0000 7ff05aea7700 1 -- 172.21.15.39:0/3097083345 wait complete. 2023-06-22T18:03:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2023-06-22T18:03:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:32 smithi039 ceph-mon[109976]: [22/Jun/2023:18:03:30] ENGINE Bus STARTING 2023-06-22T18:03:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:32 smithi039 ceph-mon[109976]: [22/Jun/2023:18:03:30] ENGINE Serving on https://172.21.15.39:7150 2023-06-22T18:03:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:32 smithi039 ceph-mon[109976]: [22/Jun/2023:18:03:30] ENGINE Bus STARTED 2023-06-22T18:03:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:32 smithi039 ceph-mon[109976]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:32.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: Generating public/private rsa key pair. 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: Your identification has been saved in /tmp/tmp211ebewc/key. 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: Your public key has been saved in /tmp/tmp211ebewc/key.pub. 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: The key fingerprint is: 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: SHA256:KcZQr5mvhfMJ3oN/KfcGaROsw+sOrv1Rcl3p5PhizBc ceph-d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: The key's randomart image is: 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: +---[RSA 3072]----+ 2023-06-22T18:03:32.946 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | . | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | . . . | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | . .. + | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | o + .o. * | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | B.S.ooo E | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | . +++=o . . | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | =.++ += o | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | +.O+o+..o | 2023-06-22T18:03:32.947 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: | ..==O= o. | 2023-06-22T18:03:32.948 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:32 smithi039 conmon[110391]: +----[SHA256]-----+ 2023-06-22T18:03:33.274 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.403+0000 7f0046c25700 1 Processor -- start 2023-06-22T18:03:33.274 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.403+0000 7f0046c25700 1 -- start start 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.403+0000 7f0046c25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f00401025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.403+0000 7f0046c25700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040102b30 con 0x7f00401021e0 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.404+0000 7f00449c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f00401025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.404+0000 7f00449c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f00401025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38584/0 (socket says 172.21.15.39:38584) 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.404+0000 7f00449c1700 1 -- 172.21.15.39:0/404620045 learned_addr learned my addr 172.21.15.39:0/404620045 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.404+0000 7f00449c1700 1 -- 172.21.15.39:0/404620045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040102c70 con 0x7f00401021e0 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.404+0000 7f00449c1700 1 --2- 172.21.15.39:0/404620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f00401025f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00340098e0 tx=0x7f0034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8e7e5ef7de84b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:33.275 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f003f7fe700 1 -- 172.21.15.39:0/404620045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0034009e80 con 0x7f00401021e0 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f003f7fe700 1 -- 172.21.15.39:0/404620045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f003400bd10 con 0x7f00401021e0 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f003f7fe700 1 -- 172.21.15.39:0/404620045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0034011ce0 con 0x7f00401021e0 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 -- 172.21.15.39:0/404620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 msgr2=0x7f00401025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 --2- 172.21.15.39:0/404620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f00401025f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00340098e0 tx=0x7f0034009250 comp rx=0 tx=0).stop 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 -- 172.21.15.39:0/404620045 shutdown_connections 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 --2- 172.21.15.39:0/404620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f00401025f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:33.276 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 -- 172.21.15.39:0/404620045 >> 172.21.15.39:0/404620045 conn(0x7f00400fd960 msgr2=0x7f00400ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 -- 172.21.15.39:0/404620045 shutdown_connections 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.405+0000 7f0046c25700 1 -- 172.21.15.39:0/404620045 wait complete. 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.406+0000 7f0046c25700 1 Processor -- start 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.406+0000 7f0046c25700 1 -- start start 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.407+0000 7f0046c25700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f0040113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.407+0000 7f0046c25700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00401135b0 con 0x7f00401021e0 2023-06-22T18:03:33.277 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.407+0000 7f00449c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f0040113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.407+0000 7f00449c1700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f0040113070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38596/0 (socket says 172.21.15.39:38596) 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.407+0000 7f00449c1700 1 -- 172.21.15.39:0/1911946218 learned_addr learned my addr 172.21.15.39:0/1911946218 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.407+0000 7f00449c1700 1 -- 172.21.15.39:0/1911946218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340094f0 con 0x7f00401021e0 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.408+0000 7f00449c1700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f0040113070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0034006b00 tx=0x7f0034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.408+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f003401a640 con 0x7f00401021e0 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.408+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f003401ac50 con 0x7f00401021e0 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.408+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00401137b0 con 0x7f00401021e0 2023-06-22T18:03:33.278 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.408+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0034022bb0 con 0x7f00401021e0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.408+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0040113bc0 con 0x7f00401021e0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.409+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f0034022490 con 0x7f00401021e0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.409+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004004b970 con 0x7f00401021e0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.409+0000 7f003d7fa700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f00300365c0 0x7f0030038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.409+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f003404a4b0 con 0x7f00401021e0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.410+0000 7f003ffff700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f00300365c0 0x7f0030038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.410+0000 7f003ffff700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f00300365c0 0x7f0030038a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f002c009990 tx=0x7f002c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:33.279 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.414+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f003401adc0 con 0x7f00401021e0 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.621+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f00400fed30 con 0x7f00300365c0 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.822+0000 7f003d7fa700 1 -- 172.21.15.39:0/1911946218 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00400fed30 con 0x7f00300365c0 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f00300365c0 msgr2=0x7f0030038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f00300365c0 0x7f0030038a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f002c009990 tx=0x7f002c006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 msgr2=0x7f0040113070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f0040113070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0034006b00 tx=0x7f0034009250 comp rx=0 tx=0).stop 2023-06-22T18:03:33.280 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 shutdown_connections 2023-06-22T18:03:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f00300365c0 0x7f0030038a70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 --2- 172.21.15.39:0/1911946218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00401021e0 0x7f0040113070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 >> 172.21.15.39:0/1911946218 conn(0x7f00400fd960 msgr2=0x7f00400fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 shutdown_connections 2023-06-22T18:03:33.281 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:32.825+0000 7f0046c25700 1 -- 172.21.15.39:0/1911946218 wait complete. 2023-06-22T18:03:33.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:33 smithi039 ceph-mon[109976]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:33.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:33 smithi039 ceph-mon[109976]: Generating ssh key... 2023-06-22T18:03:33.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:33 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:33.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:33 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:34.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:03:34.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.803+0000 7f3dea294700 1 Processor -- start 2023-06-22T18:03:34.448 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.804+0000 7f3dea294700 1 -- start start 2023-06-22T18:03:34.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.804+0000 7f3dea294700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de41047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:34.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.804+0000 7f3dea294700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4104d30 con 0x7f3de41043e0 2023-06-22T18:03:34.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.804+0000 7f3de3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de41047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:34.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.804+0000 7f3de3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de41047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38598/0 (socket says 172.21.15.39:38598) 2023-06-22T18:03:34.449 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.804+0000 7f3de3fff700 1 -- 172.21.15.39:0/1890885950 learned_addr learned my addr 172.21.15.39:0/1890885950 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.805+0000 7f3de3fff700 1 -- 172.21.15.39:0/1890885950 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4104e70 con 0x7f3de41043e0 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.805+0000 7f3de3fff700 1 --2- 172.21.15.39:0/1890885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de41047f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40098e0 tx=0x7f3dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78faf85165c94ec server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.805+0000 7f3de2ffd700 1 -- 172.21.15.39:0/1890885950 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd4009e80 con 0x7f3de41043e0 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.805+0000 7f3de2ffd700 1 -- 172.21.15.39:0/1890885950 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3dd400bd10 con 0x7f3de41043e0 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.805+0000 7f3de2ffd700 1 -- 172.21.15.39:0/1890885950 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd4011ce0 con 0x7f3de41043e0 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 -- 172.21.15.39:0/1890885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 msgr2=0x7f3de41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 --2- 172.21.15.39:0/1890885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de41047f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40098e0 tx=0x7f3dd4009250 comp rx=0 tx=0).stop 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 -- 172.21.15.39:0/1890885950 shutdown_connections 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 --2- 172.21.15.39:0/1890885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de41047f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 -- 172.21.15.39:0/1890885950 >> 172.21.15.39:0/1890885950 conn(0x7f3de4077990 msgr2=0x7f3de4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 -- 172.21.15.39:0/1890885950 shutdown_connections 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.806+0000 7f3dea294700 1 -- 172.21.15.39:0/1890885950 wait complete. 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3dea294700 1 Processor -- start 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3dea294700 1 -- start start 2023-06-22T18:03:34.451 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3dea294700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de4113060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:34.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3dea294700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41135a0 con 0x7f3de41043e0 2023-06-22T18:03:34.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3de3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de4113060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:34.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3de3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de4113060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38608/0 (socket says 172.21.15.39:38608) 2023-06-22T18:03:34.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.807+0000 7f3de3fff700 1 -- 172.21.15.39:0/2565328635 learned_addr learned my addr 172.21.15.39:0/2565328635 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:34.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3de3fff700 1 -- 172.21.15.39:0/2565328635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40094f0 con 0x7f3de41043e0 2023-06-22T18:03:34.452 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3de3fff700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de4113060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4011970 tx=0x7f3dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd401a670 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3dd401ac80 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de41137a0 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dd4022b30 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.808+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de4113bb0 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.809+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f3dd40223c0 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.809+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de404b970 con 0x7f3de41043e0 2023-06-22T18:03:34.453 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.810+0000 7f3de0ff9700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3dcc0365c0 0x7f3dcc038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.810+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3dd4049a90 con 0x7f3de41043e0 2023-06-22T18:03:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.810+0000 7f3de37fe700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3dcc0365c0 0x7f3dcc038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.811+0000 7f3de37fe700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3dcc0365c0 0x7f3dcc038a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009990 tx=0x7f3dd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:33.815+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f3dd4010b30 con 0x7f3de41043e0 2023-06-22T18:03:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.023+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3de4102380 con 0x7f3dcc0365c0 2023-06-22T18:03:34.454 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.023+0000 7f3de0ff9700 1 -- 172.21.15.39:0/2565328635 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f3de4102380 con 0x7f3dcc0365c0 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3dcc0365c0 msgr2=0x7f3dcc038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3dcc0365c0 0x7f3dcc038a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009990 tx=0x7f3dd8006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 msgr2=0x7f3de4113060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de4113060 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4011970 tx=0x7f3dd4009250 comp rx=0 tx=0).stop 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 shutdown_connections 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3dcc0365c0 0x7f3dcc038a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 --2- 172.21.15.39:0/2565328635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de41043e0 0x7f3de4113060 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:34.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 >> 172.21.15.39:0/2565328635 conn(0x7f3de4077990 msgr2=0x7f3de4101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:34.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 shutdown_connections 2023-06-22T18:03:34.456 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.026+0000 7f3dea294700 1 -- 172.21.15.39:0/2565328635 wait complete. 2023-06-22T18:03:34.456 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-06-22T18:03:34.456 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2023-06-22T18:03:34.457 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2023-06-22T18:03:35.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:35 smithi039 ceph-mon[109976]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:36.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:36 smithi039 ceph-mon[109976]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:37.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:37 smithi039 ceph-mon[109976]: Deploying cephadm binary to smithi039 2023-06-22T18:03:38.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2023-06-22T18:03:38.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.958+0000 7f51e899e700 1 Processor -- start 2023-06-22T18:03:38.325 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.958+0000 7f51e899e700 1 -- start start 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.959+0000 7f51e899e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e41047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.959+0000 7f51e899e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4104d30 con 0x7f51e41043e0 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.959+0000 7f51e259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e41047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.959+0000 7f51e259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e41047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38616/0 (socket says 172.21.15.39:38616) 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.959+0000 7f51e259c700 1 -- 172.21.15.39:0/2012803107 learned_addr learned my addr 172.21.15.39:0/2012803107 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.960+0000 7f51e259c700 1 -- 172.21.15.39:0/2012803107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4104e70 con 0x7f51e41043e0 2023-06-22T18:03:38.326 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.960+0000 7f51e259c700 1 --2- 172.21.15.39:0/2012803107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e41047f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51d40098e0 tx=0x7f51d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d1cb26139adcb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.960+0000 7f51e159a700 1 -- 172.21.15.39:0/2012803107 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d4009e80 con 0x7f51e41043e0 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.960+0000 7f51e159a700 1 -- 172.21.15.39:0/2012803107 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51d400bd10 con 0x7f51e41043e0 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.960+0000 7f51e159a700 1 -- 172.21.15.39:0/2012803107 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d4011ce0 con 0x7f51e41043e0 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 -- 172.21.15.39:0/2012803107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 msgr2=0x7f51e41047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 --2- 172.21.15.39:0/2012803107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e41047f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51d40098e0 tx=0x7f51d4009250 comp rx=0 tx=0).stop 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 -- 172.21.15.39:0/2012803107 shutdown_connections 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 --2- 172.21.15.39:0/2012803107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e41047f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:38.327 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 -- 172.21.15.39:0/2012803107 >> 172.21.15.39:0/2012803107 conn(0x7f51e4077990 msgr2=0x7f51e4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 -- 172.21.15.39:0/2012803107 shutdown_connections 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.961+0000 7f51e899e700 1 -- 172.21.15.39:0/2012803107 wait complete. 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e899e700 1 Processor -- start 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e899e700 1 -- start start 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e899e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e4113040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e899e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4113580 con 0x7f51e41043e0 2023-06-22T18:03:38.328 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e4113040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e4113040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38622/0 (socket says 172.21.15.39:38622) 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.962+0000 7f51e259c700 1 -- 172.21.15.39:0/3535331482 learned_addr learned my addr 172.21.15.39:0/3535331482 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51e259c700 1 -- 172.21.15.39:0/3535331482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d40094f0 con 0x7f51e41043e0 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51e259c700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e4113040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f51d4006b00 tx=0x7f51d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d401a640 con 0x7f51e41043e0 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51d401ac50 con 0x7f51e41043e0 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d4022bb0 con 0x7f51e41043e0 2023-06-22T18:03:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e4113780 con 0x7f51e41043e0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.963+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e4113b90 con 0x7f51e41043e0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.964+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e404b970 con 0x7f51e41043e0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.964+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f51d4022490 con 0x7f51e41043e0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.965+0000 7f51d2ffd700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f51cc0365c0 0x7f51cc038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.965+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f51d404a430 con 0x7f51e41043e0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.965+0000 7f51e1d9b700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f51cc0365c0 0x7f51cc038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.966+0000 7f51e1d9b700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f51cc0365c0 0x7f51cc038a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009990 tx=0x7f51d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:38.330 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:34.970+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f51d401adc0 con 0x7f51e41043e0 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:35.177+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}) v1 -- 0x7f51e4102310 con 0x7f51cc0365c0 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.822+0000 7f51d2ffd700 1 -- 172.21.15.39:0/3535331482 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f51e4102310 con 0x7f51cc0365c0 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f51cc0365c0 msgr2=0x7f51cc038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f51cc0365c0 0x7f51cc038a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009990 tx=0x7f51d8006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 msgr2=0x7f51e4113040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e4113040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f51d4006b00 tx=0x7f51d4009250 comp rx=0 tx=0).stop 2023-06-22T18:03:38.331 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 shutdown_connections 2023-06-22T18:03:38.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f51cc0365c0 0x7f51cc038a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:38.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 --2- 172.21.15.39:0/3535331482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51e41043e0 0x7f51e4113040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:38.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 >> 172.21.15.39:0/3535331482 conn(0x7f51e4077990 msgr2=0x7f51e4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:38.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 shutdown_connections 2023-06-22T18:03:38.332 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:37.825+0000 7f51e899e700 1 -- 172.21.15.39:0/3535331482 wait complete. 2023-06-22T18:03:38.332 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2023-06-22T18:03:38.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:38 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:38.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:38 smithi039 ceph-mon[109976]: Added host smithi039 2023-06-22T18:03:38.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:38 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:39.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-06-22T18:03:39.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.851+0000 7f1096454700 1 Processor -- start 2023-06-22T18:03:39.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.852+0000 7f1096454700 1 -- start start 2023-06-22T18:03:39.518 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.852+0000 7f1096454700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f10901025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.852+0000 7f1096454700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090102b30 con 0x7f10901021e0 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.852+0000 7f108ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f10901025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.852+0000 7f108ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f10901025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50396/0 (socket says 172.21.15.39:50396) 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.852+0000 7f108ffff700 1 -- 172.21.15.39:0/847184178 learned_addr learned my addr 172.21.15.39:0/847184178 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.853+0000 7f108ffff700 1 -- 172.21.15.39:0/847184178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090102c70 con 0x7f10901021e0 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.853+0000 7f108ffff700 1 --2- 172.21.15.39:0/847184178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f10901025f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f10800098e0 tx=0x7f1080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38be0677a43e0431 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:39.519 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.853+0000 7f108effd700 1 -- 172.21.15.39:0/847184178 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1080009e80 con 0x7f10901021e0 2023-06-22T18:03:39.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.853+0000 7f108effd700 1 -- 172.21.15.39:0/847184178 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f108000bd10 con 0x7f10901021e0 2023-06-22T18:03:39.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.853+0000 7f108effd700 1 -- 172.21.15.39:0/847184178 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1080011ce0 con 0x7f10901021e0 2023-06-22T18:03:39.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 -- 172.21.15.39:0/847184178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 msgr2=0x7f10901025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:39.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 --2- 172.21.15.39:0/847184178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f10901025f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f10800098e0 tx=0x7f1080009250 comp rx=0 tx=0).stop 2023-06-22T18:03:39.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 -- 172.21.15.39:0/847184178 shutdown_connections 2023-06-22T18:03:39.520 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 --2- 172.21.15.39:0/847184178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f10901025f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:39.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 -- 172.21.15.39:0/847184178 >> 172.21.15.39:0/847184178 conn(0x7f10900fd960 msgr2=0x7f10900ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:39.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 -- 172.21.15.39:0/847184178 shutdown_connections 2023-06-22T18:03:39.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.854+0000 7f1096454700 1 -- 172.21.15.39:0/847184178 wait complete. 2023-06-22T18:03:39.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f1096454700 1 Processor -- start 2023-06-22T18:03:39.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f1096454700 1 -- start start 2023-06-22T18:03:39.521 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f1096454700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f1090077c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:39.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f1096454700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090076080 con 0x7f10901021e0 2023-06-22T18:03:39.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f108ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f1090077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:39.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f108ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f1090077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50398/0 (socket says 172.21.15.39:50398) 2023-06-22T18:03:39.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.855+0000 7f108ffff700 1 -- 172.21.15.39:0/949498191 learned_addr learned my addr 172.21.15.39:0/949498191 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:39.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f108ffff700 1 -- 172.21.15.39:0/949498191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10800094f0 con 0x7f10901021e0 2023-06-22T18:03:39.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f108ffff700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f1090077c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1080006b00 tx=0x7f1080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f108001a640 con 0x7f10901021e0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f108001ac50 con 0x7f10901021e0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1080022bb0 con 0x7f10901021e0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1090076280 con 0x7f10901021e0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.856+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1090076690 con 0x7f10901021e0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.857+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f1080022490 con 0x7f10901021e0 2023-06-22T18:03:39.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.857+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109004b970 con 0x7f10901021e0 2023-06-22T18:03:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.858+0000 7f108cff9700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f1078036610 0x7f1078038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.858+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f108004a4b0 con 0x7f10901021e0 2023-06-22T18:03:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.858+0000 7f108f7fe700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f1078036610 0x7f1078038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.859+0000 7f108f7fe700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f1078036610 0x7f1078038ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1084009990 tx=0x7f1084006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:38.863+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f108001adc0 con 0x7f10901021e0 2023-06-22T18:03:39.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.083+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1090076e10 con 0x7f1078036610 2023-06-22T18:03:39.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.088+0000 7f108cff9700 1 -- 172.21.15.39:0/949498191 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1090076e10 con 0x7f1078036610 2023-06-22T18:03:39.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.090+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f1078036610 msgr2=0x7f1078038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:39.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f1078036610 0x7f1078038ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1084009990 tx=0x7f1084006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:39.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 msgr2=0x7f1090077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:39.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f1090077c00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1080006b00 tx=0x7f1080009250 comp rx=0 tx=0).stop 2023-06-22T18:03:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 shutdown_connections 2023-06-22T18:03:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f1078036610 0x7f1078038ac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 --2- 172.21.15.39:0/949498191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10901021e0 0x7f1090077c00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 >> 172.21.15.39:0/949498191 conn(0x7f10900fd960 msgr2=0x7f10900fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 shutdown_connections 2023-06-22T18:03:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:39.091+0000 7f1096454700 1 -- 172.21.15.39:0/949498191 wait complete. 2023-06-22T18:03:39.527 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2023-06-22T18:03:40.156 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:40 smithi039 ceph-mon[109976]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:40.156 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:40 smithi039 ceph-mon[109976]: Saving service mon spec with placement count:5 2023-06-22T18:03:40.157 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:40 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:40.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-06-22T18:03:40.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.175+0000 7fadf01e5700 1 Processor -- start 2023-06-22T18:03:40.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.175+0000 7fadf01e5700 1 -- start start 2023-06-22T18:03:40.841 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadf01e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade81025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadf01e5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8102bc0 con 0x7fade81021e0 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadedf81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade81025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadedf81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade81025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50404/0 (socket says 172.21.15.39:50404) 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadedf81700 1 -- 172.21.15.39:0/2083415988 learned_addr learned my addr 172.21.15.39:0/2083415988 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadedf81700 1 -- 172.21.15.39:0/2083415988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade81033f0 con 0x7fade81021e0 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.176+0000 7fadedf81700 1 --2- 172.21.15.39:0/2083415988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade81025f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fade40098e0 tx=0x7fade4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e00289ac0f6902 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadecf7f700 1 -- 172.21.15.39:0/2083415988 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade4009e80 con 0x7fade81021e0 2023-06-22T18:03:40.842 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadecf7f700 1 -- 172.21.15.39:0/2083415988 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fade400bd10 con 0x7fade81021e0 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadecf7f700 1 -- 172.21.15.39:0/2083415988 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade4011ce0 con 0x7fade81021e0 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadf01e5700 1 -- 172.21.15.39:0/2083415988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 msgr2=0x7fade81025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadf01e5700 1 --2- 172.21.15.39:0/2083415988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade81025f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fade40098e0 tx=0x7fade4009250 comp rx=0 tx=0).stop 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadf01e5700 1 -- 172.21.15.39:0/2083415988 shutdown_connections 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadf01e5700 1 --2- 172.21.15.39:0/2083415988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade81025f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.177+0000 7fadf01e5700 1 -- 172.21.15.39:0/2083415988 >> 172.21.15.39:0/2083415988 conn(0x7fade80fd8a0 msgr2=0x7fade80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:40.843 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.178+0000 7fadf01e5700 1 -- 172.21.15.39:0/2083415988 shutdown_connections 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.178+0000 7fadf01e5700 1 -- 172.21.15.39:0/2083415988 wait complete. 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.178+0000 7fadf01e5700 1 Processor -- start 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadf01e5700 1 -- start start 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadf01e5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade8110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadf01e5700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8111410 con 0x7fade81021e0 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadedf81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade8110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:40.844 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadedf81700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade8110ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50412/0 (socket says 172.21.15.39:50412) 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadedf81700 1 -- 172.21.15.39:0/1266556273 learned_addr learned my addr 172.21.15.39:0/1266556273 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.179+0000 7fadedf81700 1 -- 172.21.15.39:0/1266556273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40094f0 con 0x7fade81021e0 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.180+0000 7fadedf81700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade8110ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fade4011970 tx=0x7fade4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.180+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade4009e80 con 0x7fade81021e0 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.180+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fade401aaf0 con 0x7fade81021e0 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.180+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade8111610 con 0x7fade81021e0 2023-06-22T18:03:40.845 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.180+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fade4022a90 con 0x7fade81021e0 2023-06-22T18:03:40.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.180+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade8111a20 con 0x7fade81021e0 2023-06-22T18:03:40.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.181+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade805f060 con 0x7fade81021e0 2023-06-22T18:03:40.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.181+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7fade4022400 con 0x7fade81021e0 2023-06-22T18:03:40.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.182+0000 7fadde7fc700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7fadd40365c0 0x7fadd4038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:40.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.182+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fade404a460 con 0x7fade81021e0 2023-06-22T18:03:40.846 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.182+0000 7faded780700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7fadd40365c0 0x7fadd4038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.183+0000 7faded780700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7fadd40365c0 0x7fadd4038a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009990 tx=0x7fadd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.187+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fade40219f0 con 0x7fade81021e0 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.354+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fade40219f0 con 0x7fade81021e0 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.414+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fade80fecf0 con 0x7fadd40365c0 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.420+0000 7fadde7fc700 1 -- 172.21.15.39:0/1266556273 <== mgr.14120 v2:172.21.15.39:6800/349518747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fade80fecf0 con 0x7fadd40365c0 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7fadd40365c0 msgr2=0x7fadd4038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7fadd40365c0 0x7fadd4038a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009990 tx=0x7fadd8006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:40.847 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 msgr2=0x7fade8110ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade8110ed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fade4011970 tx=0x7fade4009250 comp rx=0 tx=0).stop 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 shutdown_connections 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7fadd40365c0 0x7fadd4038a70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 --2- 172.21.15.39:0/1266556273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade81021e0 0x7fade8110ed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 >> 172.21.15.39:0/1266556273 conn(0x7fade80fd8a0 msgr2=0x7fade80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 shutdown_connections 2023-06-22T18:03:40.848 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:40.422+0000 7fadf01e5700 1 -- 172.21.15.39:0/1266556273 wait complete. 2023-06-22T18:03:41.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:41 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:41.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:41 smithi039 ceph-mon[109976]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:41.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:41 smithi039 ceph-mon[109976]: Saving service mgr spec with placement count:2 2023-06-22T18:03:41.655 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:41 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:41.655 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:41 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:42.115 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.374+0000 7f616b0e4700 1 Processor -- start 2023-06-22T18:03:42.115 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.374+0000 7f616b0e4700 1 -- start start 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.375+0000 7f616b0e4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f61641050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.375+0000 7f616b0e4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164070e60 con 0x7f6164102d00 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.375+0000 7f6168e80700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f61641050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.375+0000 7f6168e80700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f61641050e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50422/0 (socket says 172.21.15.39:50422) 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.375+0000 7f6168e80700 1 -- 172.21.15.39:0/3731766084 learned_addr learned my addr 172.21.15.39:0/3731766084 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.375+0000 7f6168e80700 1 -- 172.21.15.39:0/3731766084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164070fa0 con 0x7f6164102d00 2023-06-22T18:03:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.376+0000 7f6168e80700 1 --2- 172.21.15.39:0/3731766084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f61641050e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f61580098e0 tx=0x7f6158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ffa54cb77df191 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.376+0000 7f61637fe700 1 -- 172.21.15.39:0/3731766084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6158009e80 con 0x7f6164102d00 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.376+0000 7f61637fe700 1 -- 172.21.15.39:0/3731766084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f615800bd30 con 0x7f6164102d00 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.376+0000 7f61637fe700 1 -- 172.21.15.39:0/3731766084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6158011ca0 con 0x7f6164102d00 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 -- 172.21.15.39:0/3731766084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 msgr2=0x7f61641050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 --2- 172.21.15.39:0/3731766084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f61641050e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f61580098e0 tx=0x7f6158009250 comp rx=0 tx=0).stop 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 -- 172.21.15.39:0/3731766084 shutdown_connections 2023-06-22T18:03:42.117 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 --2- 172.21.15.39:0/3731766084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f61641050e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:42.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 -- 172.21.15.39:0/3731766084 >> 172.21.15.39:0/3731766084 conn(0x7f61640fc7a0 msgr2=0x7f61640febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:42.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 -- 172.21.15.39:0/3731766084 shutdown_connections 2023-06-22T18:03:42.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.377+0000 7f616b0e4700 1 -- 172.21.15.39:0/3731766084 wait complete. 2023-06-22T18:03:42.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.378+0000 7f616b0e4700 1 Processor -- start 2023-06-22T18:03:42.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.378+0000 7f616b0e4700 1 -- start start 2023-06-22T18:03:42.118 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.378+0000 7f616b0e4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f6164102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.378+0000 7f616b0e4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164100c10 con 0x7f6164102d00 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f6168e80700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f6164102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f6168e80700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f6164102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50424/0 (socket says 172.21.15.39:50424) 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f6168e80700 1 -- 172.21.15.39:0/2651919272 learned_addr learned my addr 172.21.15.39:0/2651919272 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f6168e80700 1 -- 172.21.15.39:0/2651919272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580094f0 con 0x7f6164102d00 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f6168e80700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f6164102790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f615801a760 con 0x7f6164102d00 2023-06-22T18:03:42.119 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164100e10 con 0x7f6164102d00 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f615801ad90 con 0x7f6164102d00 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.379+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6158022c60 con 0x7f6164102d00 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.380+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6164101220 con 0x7f6164102d00 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.381+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f615802c440 con 0x7f6164102d00 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.381+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616404b970 con 0x7f6164102d00 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.381+0000 7f61617fa700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f61540365c0 0x7f6154038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:42.120 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.381+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f615804a400 con 0x7f6164102d00 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.382+0000 7f6163fff700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f61540365c0 0x7f6154038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.382+0000 7f6163fff700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f61540365c0 0x7f6154038a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6150009990 tx=0x7f6150006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.387+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f61580217b0 con 0x7f6164102d00 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.610+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f6164070fa0 con 0x7f6164102d00 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.616+0000 7f61617fa700 1 -- 172.21.15.39:0/2651919272 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f6164070fa0 con 0x7f6164102d00 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f61540365c0 msgr2=0x7f6154038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:42.121 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f61540365c0 0x7f6154038a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6150009990 tx=0x7f6150006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 msgr2=0x7f6164102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f6164102790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c00 tx=0x7f6158009250 comp rx=0 tx=0).stop 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 shutdown_connections 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f61540365c0 0x7f6154038a70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 --2- 172.21.15.39:0/2651919272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6164102d00 0x7f6164102790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.619+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 >> 172.21.15.39:0/2651919272 conn(0x7f61640fc7a0 msgr2=0x7f61640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.620+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 shutdown_connections 2023-06-22T18:03:42.122 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:41.620+0000 7f616b0e4700 1 -- 172.21.15.39:0/2651919272 wait complete. 2023-06-22T18:03:42.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:42 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2651919272' entity='client.admin' 2023-06-22T18:03:43.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.670+0000 7f4a5a175700 1 Processor -- start 2023-06-22T18:03:43.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.671+0000 7f4a5a175700 1 -- start start 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.671+0000 7f4a5a175700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.671+0000 7f4a5a175700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54102bc0 con 0x7f4a541021e0 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.671+0000 7f4a537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.671+0000 7f4a537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50428/0 (socket says 172.21.15.39:50428) 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.671+0000 7f4a537fe700 1 -- 172.21.15.39:0/1392022974 learned_addr learned my addr 172.21.15.39:0/1392022974 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.672+0000 7f4a537fe700 1 -- 172.21.15.39:0/1392022974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a541033f0 con 0x7f4a541021e0 2023-06-22T18:03:43.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.672+0000 7f4a537fe700 1 --2- 172.21.15.39:0/1392022974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541025f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a440098e0 tx=0x7f4a44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28f486f8f81226 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.672+0000 7f4a527fc700 1 -- 172.21.15.39:0/1392022974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a44009e80 con 0x7f4a541021e0 2023-06-22T18:03:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.672+0000 7f4a527fc700 1 -- 172.21.15.39:0/1392022974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4400bd30 con 0x7f4a541021e0 2023-06-22T18:03:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.672+0000 7f4a527fc700 1 -- 172.21.15.39:0/1392022974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a44011ca0 con 0x7f4a541021e0 2023-06-22T18:03:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 -- 172.21.15.39:0/1392022974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 msgr2=0x7f4a541025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:43.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 --2- 172.21.15.39:0/1392022974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541025f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4a440098e0 tx=0x7f4a44009250 comp rx=0 tx=0).stop 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 -- 172.21.15.39:0/1392022974 shutdown_connections 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 --2- 172.21.15.39:0/1392022974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541025f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 -- 172.21.15.39:0/1392022974 >> 172.21.15.39:0/1392022974 conn(0x7f4a540fd8a0 msgr2=0x7f4a540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 -- 172.21.15.39:0/1392022974 shutdown_connections 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.673+0000 7f4a5a175700 1 -- 172.21.15.39:0/1392022974 wait complete. 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a5a175700 1 Processor -- start 2023-06-22T18:03:43.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a5a175700 1 -- start start 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a5a175700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a5a175700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54113630 con 0x7f4a541021e0 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a537fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50434/0 (socket says 172.21.15.39:50434) 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.674+0000 7f4a537fe700 1 -- 172.21.15.39:0/2134881683 learned_addr learned my addr 172.21.15.39:0/2134881683 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.675+0000 7f4a537fe700 1 -- 172.21.15.39:0/2134881683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440094f0 con 0x7f4a541021e0 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.675+0000 7f4a537fe700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541130f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a44006fd0 tx=0x7f4a44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:43.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.675+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a4401a710 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.675+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4401ad40 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.675+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a54113830 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.675+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a44022c90 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.676+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a54113c40 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.676+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f4a44022500 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.676+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5405f060 con 0x7f4a541021e0 2023-06-22T18:03:43.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.677+0000 7f4a3bfff700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f4a3c0365c0 0x7f4a3c038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.677+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4a4404a5f0 con 0x7f4a541021e0 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.677+0000 7f4a52ffd700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f4a3c0365c0 0x7f4a3c038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.678+0000 7f4a52ffd700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f4a3c0365c0 0x7f4a3c038a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009990 tx=0x7f4a48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.682+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f4a44021bd0 con 0x7f4a541021e0 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.893+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f4a5404b970 con 0x7f4a541021e0 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.896+0000 7f4a3bfff700 1 -- 172.21.15.39:0/2134881683 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f4a440213a0 con 0x7f4a541021e0 2023-06-22T18:03:43.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f4a3c0365c0 msgr2=0x7f4a3c038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f4a3c0365c0 0x7f4a3c038a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009990 tx=0x7f4a48006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 msgr2=0x7f4a541130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541130f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a44006fd0 tx=0x7f4a44009250 comp rx=0 tx=0).stop 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 shutdown_connections 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f4a3c0365c0 0x7f4a3c038a70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 --2- 172.21.15.39:0/2134881683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a541021e0 0x7f4a541130f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 >> 172.21.15.39:0/2134881683 conn(0x7f4a540fd8a0 msgr2=0x7f4a540fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:43.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.899+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 shutdown_connections 2023-06-22T18:03:43.499 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:42.900+0000 7f4a5a175700 1 -- 172.21.15.39:0/2134881683 wait complete. 2023-06-22T18:03:43.499 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2023-06-22T18:03:44.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:43 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2134881683' entity='client.admin' 2023-06-22T18:03:44.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:43 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:45.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:44 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/800284202' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-06-22T18:03:45.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:44 smithi039 ceph-mon[109976]: from='mgr.14120 172.21.15.39:0/3581836072' entity='mgr.x' 2023-06-22T18:03:45.176 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:45 smithi039 conmon[110391]: ignoring --setuser ceph since I am not root 2023-06-22T18:03:45.176 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:45 smithi039 conmon[110391]: ignoring --setgroup ceph since I am not root 2023-06-22T18:03:45.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.084+0000 7f160874a700 1 Processor -- start 2023-06-22T18:03:45.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.084+0000 7f160874a700 1 -- start start 2023-06-22T18:03:45.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.084+0000 7f160874a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f16000760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:45.367 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.084+0000 7f160874a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600076620 con 0x7f1600077c70 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.085+0000 7f16064e6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f16000760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.085+0000 7f16064e6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f16000760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50440/0 (socket says 172.21.15.39:50440) 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.085+0000 7f16064e6700 1 -- 172.21.15.39:0/3323876444 learned_addr learned my addr 172.21.15.39:0/3323876444 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.085+0000 7f16064e6700 1 -- 172.21.15.39:0/3323876444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600076760 con 0x7f1600077c70 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.085+0000 7f16064e6700 1 --2- 172.21.15.39:0/3323876444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f16000760e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0098e0 tx=0x7f15fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb374ba609beef9 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f16054e4700 1 -- 172.21.15.39:0/3323876444 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15fc009e80 con 0x7f1600077c70 2023-06-22T18:03:45.368 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f16054e4700 1 -- 172.21.15.39:0/3323876444 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc00bd30 con 0x7f1600077c70 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f16054e4700 1 -- 172.21.15.39:0/3323876444 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15fc011ca0 con 0x7f1600077c70 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f160874a700 1 -- 172.21.15.39:0/3323876444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 msgr2=0x7f16000760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f160874a700 1 --2- 172.21.15.39:0/3323876444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f16000760e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0098e0 tx=0x7f15fc009250 comp rx=0 tx=0).stop 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f160874a700 1 -- 172.21.15.39:0/3323876444 shutdown_connections 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f160874a700 1 --2- 172.21.15.39:0/3323876444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f16000760e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.086+0000 7f160874a700 1 -- 172.21.15.39:0/3323876444 >> 172.21.15.39:0/3323876444 conn(0x7f16000fd8a0 msgr2=0x7f16000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:45.369 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.087+0000 7f160874a700 1 -- 172.21.15.39:0/3323876444 shutdown_connections 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.087+0000 7f160874a700 1 -- 172.21.15.39:0/3323876444 wait complete. 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.087+0000 7f160874a700 1 Processor -- start 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f160874a700 1 -- start start 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f160874a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f1600113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f160874a700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001135d0 con 0x7f1600077c70 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f16064e6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f1600113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f16064e6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f1600113090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50446/0 (socket says 172.21.15.39:50446) 2023-06-22T18:03:45.370 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f16064e6700 1 -- 172.21.15.39:0/800284202 learned_addr learned my addr 172.21.15.39:0/800284202 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.088+0000 7f16064e6700 1 -- 172.21.15.39:0/800284202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0094f0 con 0x7f1600077c70 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.089+0000 7f16064e6700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f1600113090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f15fc000c00 tx=0x7f15fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.089+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15fc01a760 con 0x7f1600077c70 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.089+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15fc01ad90 con 0x7f1600077c70 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.089+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16001137d0 con 0x7f1600077c70 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.089+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15fc022c60 con 0x7f1600077c70 2023-06-22T18:03:45.371 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.089+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1600113be0 con 0x7f1600077c70 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.090+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 8) v1 ==== 43219+0+0 (secure 0 0 0) 0x7f15fc022570 con 0x7f1600077c70 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.090+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160004b970 con 0x7f1600077c70 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.091+0000 7f15f6ffd700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f15ec0365c0 0x7f15ec038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.091+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f15fc04a600 con 0x7f1600077c70 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.091+0000 7f1605ce5700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f15ec0365c0 0x7f15ec038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.092+0000 7f1605ce5700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f15ec0365c0 0x7f15ec038a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009990 tx=0x7f15f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.095+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f15fc02ad60 con 0x7f1600077c70 2023-06-22T18:03:45.372 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.362+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f1600108590 con 0x7f1600077c70 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.906+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 9) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f15fc02c960 con 0x7f1600077c70 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.907+0000 7f15f6ffd700 1 -- 172.21.15.39:0/800284202 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f15fc0217b0 con 0x7f1600077c70 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.910+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f15ec0365c0 msgr2=0x7f15ec038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.910+0000 7f160874a700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f15ec0365c0 0x7f15ec038a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009990 tx=0x7f15f0006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.910+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 msgr2=0x7f1600113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.910+0000 7f160874a700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f1600113090 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f15fc000c00 tx=0x7f15fc009250 comp rx=0 tx=0).stop 2023-06-22T18:03:45.373 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.911+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 shutdown_connections 2023-06-22T18:03:45.374 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.911+0000 7f160874a700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f15ec0365c0 0x7f15ec038a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:45.374 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.911+0000 7f160874a700 1 --2- 172.21.15.39:0/800284202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1600077c70 0x7f1600113090 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:45.374 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.911+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 >> 172.21.15.39:0/800284202 conn(0x7f16000fd8a0 msgr2=0x7f16000fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:45.374 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.911+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 shutdown_connections 2023-06-22T18:03:45.374 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:44.911+0000 7f160874a700 1 -- 172.21.15.39:0/800284202 wait complete. 2023-06-22T18:03:46.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:45 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/800284202' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-06-22T18:03:46.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:45 smithi039 ceph-mon[109976]: mgrmap e9: x(active, since 16s) 2023-06-22T18:03:46.746 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:46.747 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-06-22T18:03:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-22T18:03:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-06-22T18:03:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-22T18:03:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.041+0000 7ff02618b700 1 Processor -- start 2023-06-22T18:03:46.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.042+0000 7ff02618b700 1 -- start start 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.042+0000 7ff02618b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff0201025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.042+0000 7ff02618b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020102bc0 con 0x7ff0201021e0 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.042+0000 7ff01f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff0201025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff0201025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54118/0 (socket says 172.21.15.39:54118) 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01f7fe700 1 -- 172.21.15.39:0/1176138547 learned_addr learned my addr 172.21.15.39:0/1176138547 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01f7fe700 1 -- 172.21.15.39:0/1176138547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0201033f0 con 0x7ff0201021e0 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01f7fe700 1 --2- 172.21.15.39:0/1176138547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff0201025f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0100098e0 tx=0x7ff010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6a32bd3aab6917 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:46.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01e7fc700 1 -- 172.21.15.39:0/1176138547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff010009e80 con 0x7ff0201021e0 2023-06-22T18:03:46.750 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01e7fc700 1 -- 172.21.15.39:0/1176138547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff01000bd30 con 0x7ff0201021e0 2023-06-22T18:03:46.750 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.043+0000 7ff01e7fc700 1 -- 172.21.15.39:0/1176138547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff010011ca0 con 0x7ff0201021e0 2023-06-22T18:03:46.750 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 -- 172.21.15.39:0/1176138547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 msgr2=0x7ff0201025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:46.753 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 --2- 172.21.15.39:0/1176138547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff0201025f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0100098e0 tx=0x7ff010009250 comp rx=0 tx=0).stop 2023-06-22T18:03:46.753 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 -- 172.21.15.39:0/1176138547 shutdown_connections 2023-06-22T18:03:46.753 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 --2- 172.21.15.39:0/1176138547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff0201025f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:46.753 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 -- 172.21.15.39:0/1176138547 >> 172.21.15.39:0/1176138547 conn(0x7ff0200fd8a0 msgr2=0x7ff0200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 -- 172.21.15.39:0/1176138547 shutdown_connections 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.044+0000 7ff02618b700 1 -- 172.21.15.39:0/1176138547 wait complete. 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.045+0000 7ff02618b700 1 Processor -- start 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.045+0000 7ff02618b700 1 -- start start 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.045+0000 7ff02618b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff020113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.046+0000 7ff02618b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020113670 con 0x7ff0201021e0 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.046+0000 7ff01f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff020113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:46.754 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.046+0000 7ff01f7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff020113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54132/0 (socket says 172.21.15.39:54132) 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.046+0000 7ff01f7fe700 1 -- 172.21.15.39:0/4142633500 learned_addr learned my addr 172.21.15.39:0/4142633500 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.046+0000 7ff01f7fe700 1 -- 172.21.15.39:0/4142633500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0100094f0 con 0x7ff0201021e0 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.046+0000 7ff01f7fe700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff020113130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff010011930 tx=0x7ff010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.047+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff01001a790 con 0x7ff0201021e0 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.047+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff01001adc0 con 0x7ff0201021e0 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.047+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff020113870 con 0x7ff0201021e0 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.047+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff010022c10 con 0x7ff0201021e0 2023-06-22T18:03:46.755 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.047+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff020113c80 con 0x7ff0201021e0 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.048+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) v1 ==== 43232+0+0 (secure 0 0 0) 0x7ff0100224c0 con 0x7ff0201021e0 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.049+0000 7ff007fff700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 0x7ff008038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.049+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff02005f060 con 0x7ff0201021e0 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.049+0000 7ff01effd700 1 -- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 msgr2=0x7ff008038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.049+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff0100497b0 con 0x7ff0201021e0 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.049+0000 7ff01effd700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 0x7ff008038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:03:46.756 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.054+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff01001e020 con 0x7ff0201021e0 2023-06-22T18:03:46.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.250+0000 7ff01effd700 1 -- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 msgr2=0x7ff008038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:46.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.250+0000 7ff01effd700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 0x7ff008038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-22T18:03:46.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.328+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff020114750 con 0x7ff0201021e0 2023-06-22T18:03:46.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.329+0000 7ff007fff700 1 -- 172.21.15.39:0/4142633500 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7ff010020e60 con 0x7ff0201021e0 2023-06-22T18:03:46.757 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 msgr2=0x7ff008038ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 0x7ff008038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 msgr2=0x7ff020113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff020113130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff010011930 tx=0x7ff010009250 comp rx=0 tx=0).stop 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 shutdown_connections 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7ff008036610 0x7ff008038ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 --2- 172.21.15.39:0/4142633500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0201021e0 0x7ff020113130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:46.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 >> 172.21.15.39:0/4142633500 conn(0x7ff0200fd8a0 msgr2=0x7ff0200fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:46.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 shutdown_connections 2023-06-22T18:03:46.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:46.331+0000 7ff02618b700 1 -- 172.21.15.39:0/4142633500 wait complete. 2023-06-22T18:03:46.759 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2023-06-22T18:03:46.759 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 9... 2023-06-22T18:03:46.914 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:46 smithi039 conmon[110391]: 2023-06-22T18:03:46.659+0000 7f197e7581c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T18:03:46.914 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:46 smithi039 conmon[110391]: 2023-06-22T18:03:46.887+0000 7f197e7581c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T18:03:47.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:46 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/4142633500' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-22T18:03:47.195 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:47 smithi039 conmon[110391]: 2023-06-22T18:03:47.013+0000 7f197e7581c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T18:03:47.568 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:47 smithi039 conmon[110391]: 2023-06-22T18:03:47.301+0000 7f197e7581c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:03:47.877 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:47 smithi039 conmon[110391]: 2023-06-22T18:03:47.565+0000 7f197e7581c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T18:03:47.877 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:47 smithi039 conmon[110391]: 2023-06-22T18:03:47.764+0000 7f197e7581c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T18:03:48.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:47 smithi039 conmon[110391]: 2023-06-22T18:03:47.873+0000 7f197e7581c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T18:03:48.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:48 smithi039 conmon[110391]: 2023-06-22T18:03:48.662+0000 7f197e7581c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T18:03:49.694 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:49 smithi039 conmon[110391]: 2023-06-22T18:03:49.299+0000 7f197e7581c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T18:03:49.694 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:49 smithi039 conmon[110391]: 2023-06-22T18:03:49.422+0000 7f197e7581c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T18:03:49.695 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:49 smithi039 conmon[110391]: 2023-06-22T18:03:49.546+0000 7f197e7581c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T18:03:50.694 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:50 smithi039 conmon[110391]: 2023-06-22T18:03:50.361+0000 7f197e7581c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T18:03:50.694 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:50 smithi039 conmon[110391]: 2023-06-22T18:03:50.589+0000 7f197e7581c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:03:51.016 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:50 smithi039 conmon[110391]: 2023-06-22T18:03:50.710+0000 7f197e7581c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T18:03:51.016 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:50 smithi039 conmon[110391]: 2023-06-22T18:03:50.887+0000 7f197e7581c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T18:03:52.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:51 smithi039 conmon[110391]: 2023-06-22T18:03:51.011+0000 7f197e7581c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T18:03:52.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:52 smithi039 conmon[110391]: 2023-06-22T18:03:52.149+0000 7f197e7581c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T18:03:52.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:52 smithi039 conmon[110391]: 2023-06-22T18:03:52.269+0000 7f197e7581c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T18:03:52.445 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:52 smithi039 conmon[110391]: 2023-06-22T18:03:52.388+0000 7f197e7581c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T18:03:52.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:52 smithi039 conmon[110391]: 2023-06-22T18:03:52.588+0000 7f197e7581c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T18:03:52.945 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:52 smithi039 conmon[110391]: 2023-06-22T18:03:52.720+0000 7f197e7581c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T18:03:53.599 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:53 smithi039 conmon[110391]: 2023-06-22T18:03:53.234+0000 7f197e7581c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T18:03:53.600 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:53 smithi039 conmon[110391]: 2023-06-22T18:03:53.354+0000 7f197e7581c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T18:03:53.600 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:53 smithi039 conmon[110391]: 2023-06-22T18:03:53.475+0000 7f197e7581c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T18:03:53.944 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:03:53 smithi039 conmon[110391]: 2023-06-22T18:03:53.595+0000 7f197e7581c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T18:03:53.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: Active manager daemon x restarted 2023-06-22T18:03:53.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: Activating manager daemon x 2023-06-22T18:03:53.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: osdmap e3: 0 total, 0 up, 0 in 2023-06-22T18:03:53.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: mgrmap e10: x(active, starting, since 0.00785089s) 2023-06-22T18:03:53.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:03:53.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-22T18:03:53.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-22T18:03:53.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-22T18:03:53.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-22T18:03:53.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: Manager daemon x is now available 2023-06-22T18:03:53.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:53.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:03:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-22T18:03:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-22T18:03:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:54 smithi039 ceph-mon[109976]: [22/Jun/2023:18:03:53] ENGINE Bus STARTING 2023-06-22T18:03:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:03:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:54 smithi039 ceph-mon[109976]: mgrmap e11: x(active, since 1.00933s) 2023-06-22T18:03:55.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2023-06-22T18:03:55.044 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.314+0000 7f341d3c8700 1 Processor -- start 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.314+0000 7f341d3c8700 1 -- start start 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.314+0000 7f341d3c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f34181025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.314+0000 7f341d3c8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418102bc0 con 0x7f34181021e0 2023-06-22T18:03:55.045 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.315+0000 7f3416ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f34181025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.315+0000 7f3416ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f34181025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54140/0 (socket says 172.21.15.39:54140) 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.315+0000 7f3416ffd700 1 -- 172.21.15.39:0/3879441747 learned_addr learned my addr 172.21.15.39:0/3879441747 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.315+0000 7f3416ffd700 1 -- 172.21.15.39:0/3879441747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34181033f0 con 0x7f34181021e0 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.315+0000 7f3416ffd700 1 --2- 172.21.15.39:0/3879441747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f34181025f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34080098e0 tx=0x7f3408009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd8da93988a51a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f3415ffb700 1 -- 172.21.15.39:0/3879441747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3408009e80 con 0x7f34181021e0 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f3415ffb700 1 -- 172.21.15.39:0/3879441747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f340800bd30 con 0x7f34181021e0 2023-06-22T18:03:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f3415ffb700 1 -- 172.21.15.39:0/3879441747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3408011ca0 con 0x7f34181021e0 2023-06-22T18:03:55.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 -- 172.21.15.39:0/3879441747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 msgr2=0x7f34181025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:55.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3879441747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f34181025f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34080098e0 tx=0x7f3408009250 comp rx=0 tx=0).stop 2023-06-22T18:03:55.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 -- 172.21.15.39:0/3879441747 shutdown_connections 2023-06-22T18:03:55.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3879441747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f34181025f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:55.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 -- 172.21.15.39:0/3879441747 >> 172.21.15.39:0/3879441747 conn(0x7f34180fd8a0 msgr2=0x7f34180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:55.047 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 -- 172.21.15.39:0/3879441747 shutdown_connections 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.316+0000 7f341d3c8700 1 -- 172.21.15.39:0/3879441747 wait complete. 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.317+0000 7f341d3c8700 1 Processor -- start 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.317+0000 7f341d3c8700 1 -- start start 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f341d3c8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f3418110f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f341d3c8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418111450 con 0x7f34181021e0 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f3416ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f3418110f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:55.048 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f3416ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f3418110f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54154/0 (socket says 172.21.15.39:54154) 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f3416ffd700 1 -- 172.21.15.39:0/3961501194 learned_addr learned my addr 172.21.15.39:0/3961501194 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f3416ffd700 1 -- 172.21.15.39:0/3961501194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080094f0 con 0x7f34181021e0 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.318+0000 7f3416ffd700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f3418110f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3408006fd0 tx=0x7f3408009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.319+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f340801a710 con 0x7f34181021e0 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.319+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f340801ad40 con 0x7f34181021e0 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.319+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3408022c90 con 0x7f34181021e0 2023-06-22T18:03:55.049 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.319+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3418111650 con 0x7f34181021e0 2023-06-22T18:03:55.050 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.319+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3418111a60 con 0x7f34181021e0 2023-06-22T18:03:55.050 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.320+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 9) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f3408022500 con 0x7f34181021e0 2023-06-22T18:03:55.050 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.320+0000 7f33ff7fe700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:55.050 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.320+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 --> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34000391d0 con 0x7f3400036610 2023-06-22T18:03:55.050 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.320+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f340804a660 con 0x7f34181021e0 2023-06-22T18:03:55.050 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.320+0000 7f34167fc700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.321+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.522+0000 7f34167fc700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.522+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.923+0000 7f34167fc700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:47.923+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:48.724+0000 7f34167fc700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:55.051 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:48.724+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:50.325+0000 7f34167fc700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:50.325+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:53.527+0000 7f34167fc700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.39:6800/349518747 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:53.527+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:53.603+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mgrmap(e 10) v1 ==== 42999+0+0 (secure 0 0 0) 0x7f340802c960 con 0x7f34181021e0 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:53.604+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 msgr2=0x7f3400038ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:53.604+0000 7f33ff7fe700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:55.052 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.604+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 11) v1 ==== 43126+0+0 (secure 0 0 0) 0x7f340802c440 con 0x7f34181021e0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.604+0000 7f33ff7fe700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3400039fd0 0x7f340003c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.604+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34000391d0 con 0x7f3400039fd0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.605+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3400039fd0 0x7f340003c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.606+0000 7f34167fc700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3400039fd0 0x7f340003c3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f340c0039d0 tx=0x7f340c009350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.607+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f34000391d0 con 0x7f3400039fd0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.613+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f341805f060 con 0x7f3400039fd0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.613+0000 7f33ff7fe700 1 -- 172.21.15.39:0/3961501194 <== mgr.14152 v2:172.21.15.39:6800/2143792276 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f341805f060 con 0x7f3400039fd0 2023-06-22T18:03:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3400039fd0 msgr2=0x7f340003c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3400039fd0 0x7f340003c3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f340c0039d0 tx=0x7f340c009350 comp rx=0 tx=0).stop 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 msgr2=0x7f3418110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f3418110f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3408006fd0 tx=0x7f3408009250 comp rx=0 tx=0).stop 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 shutdown_connections 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3400039fd0 0x7f340003c3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34181021e0 0x7f3418110f10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:55.054 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 --2- 172.21.15.39:0/3961501194 >> [v2:172.21.15.39:6800/349518747,v1:172.21.15.39:6801/349518747] conn(0x7f3400036610 0x7f3400038ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:55.055 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 >> 172.21.15.39:0/3961501194 conn(0x7f34180fd8a0 msgr2=0x7f34180fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:55.055 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 shutdown_connections 2023-06-22T18:03:55.055 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:54.614+0000 7f341d3c8700 1 -- 172.21.15.39:0/3961501194 wait complete. 2023-06-22T18:03:55.055 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 9 is available 2023-06-22T18:03:55.055 INFO:teuthology.orchestra.run.smithi039.stdout:Generating a dashboard self-signed certificate... 2023-06-22T18:03:55.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:55 smithi039 ceph-mon[109976]: [22/Jun/2023:18:03:54] ENGINE Serving on https://172.21.15.39:7150 2023-06-22T18:03:55.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:55 smithi039 ceph-mon[109976]: [22/Jun/2023:18:03:54] ENGINE Bus STARTED 2023-06-22T18:03:55.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:55 smithi039 ceph-mon[109976]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-22T18:03:55.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:55 smithi039 ceph-mon[109976]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-22T18:03:56.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-06-22T18:03:56.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.612+0000 7f4c6fc63700 1 Processor -- start 2023-06-22T18:03:56.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.613+0000 7f4c6fc63700 1 -- start start 2023-06-22T18:03:56.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.613+0000 7f4c6fc63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c680760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:56.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.613+0000 7f4c6fc63700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68076600 con 0x7f4c68077c50 2023-06-22T18:03:56.522 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.613+0000 7f4c6d9ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c680760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.613+0000 7f4c6d9ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c680760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54208/0 (socket says 172.21.15.39:54208) 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.613+0000 7f4c6d9ff700 1 -- 172.21.15.39:0/3896835502 learned_addr learned my addr 172.21.15.39:0/3896835502 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.614+0000 7f4c6d9ff700 1 -- 172.21.15.39:0/3896835502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c68076740 con 0x7f4c68077c50 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.614+0000 7f4c6d9ff700 1 --2- 172.21.15.39:0/3896835502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c680760c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c640098e0 tx=0x7f4c64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74476c3029039564 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.614+0000 7f4c6c9fd700 1 -- 172.21.15.39:0/3896835502 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c64009e80 con 0x7f4c68077c50 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.614+0000 7f4c6c9fd700 1 -- 172.21.15.39:0/3896835502 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c6400bd30 con 0x7f4c68077c50 2023-06-22T18:03:56.523 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.614+0000 7f4c6c9fd700 1 -- 172.21.15.39:0/3896835502 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c64011ca0 con 0x7f4c68077c50 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 -- 172.21.15.39:0/3896835502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 msgr2=0x7f4c680760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 --2- 172.21.15.39:0/3896835502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c680760c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4c640098e0 tx=0x7f4c64009250 comp rx=0 tx=0).stop 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 -- 172.21.15.39:0/3896835502 shutdown_connections 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 --2- 172.21.15.39:0/3896835502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c680760c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 -- 172.21.15.39:0/3896835502 >> 172.21.15.39:0/3896835502 conn(0x7f4c680fd8a0 msgr2=0x7f4c680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 -- 172.21.15.39:0/3896835502 shutdown_connections 2023-06-22T18:03:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.615+0000 7f4c6fc63700 1 -- 172.21.15.39:0/3896835502 wait complete. 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.616+0000 7f4c6fc63700 1 Processor -- start 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.616+0000 7f4c6fc63700 1 -- start start 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.616+0000 7f4c6fc63700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c68113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.616+0000 7f4c6fc63700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681135d0 con 0x7f4c68077c50 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c6d9ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c68113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c6d9ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c68113090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49558/0 (socket says 172.21.15.39:49558) 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c6d9ff700 1 -- 172.21.15.39:0/2113016012 learned_addr learned my addr 172.21.15.39:0/2113016012 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:56.525 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c6d9ff700 1 -- 172.21.15.39:0/2113016012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640094f0 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c6d9ff700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c68113090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c64006fd0 tx=0x7f4c64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c6401a790 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c6401adc0 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c681137d0 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.617+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c64019ce0 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.618+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68113be0 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.619+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6804b970 con 0x7f4c68077c50 2023-06-22T18:03:56.526 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.619+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 11) v1 ==== 43126+0+0 (secure 0 0 0) 0x7f4c640195c0 con 0x7f4c68077c50 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.619+0000 7f4c5e7fc700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f4c540364f0 0x7f4c540389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.619+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4c64049ac0 con 0x7f4c68077c50 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.619+0000 7f4c6d1fe700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f4c540364f0 0x7f4c540389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.620+0000 7f4c6d1fe700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f4c540364f0 0x7f4c540389a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009990 tx=0x7f4c58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.624+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f4c6401a8f0 con 0x7f4c68077c50 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.659+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f4c640223c0 con 0x7f4c68077c50 2023-06-22T18:03:56.527 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:55.845+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c680fed00 con 0x7f4c540364f0 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.143+0000 7f4c5e7fc700 1 -- 172.21.15.39:0/2113016012 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f4c680fed00 con 0x7f4c540364f0 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f4c540364f0 msgr2=0x7f4c540389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f4c540364f0 0x7f4c540389a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009990 tx=0x7f4c58006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 msgr2=0x7f4c68113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c68113090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c64006fd0 tx=0x7f4c64009250 comp rx=0 tx=0).stop 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 shutdown_connections 2023-06-22T18:03:56.528 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f4c540364f0 0x7f4c540389a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:56.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 --2- 172.21.15.39:0/2113016012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c68077c50 0x7f4c68113090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:56.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.145+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 >> 172.21.15.39:0/2113016012 conn(0x7f4c680fd8a0 msgr2=0x7f4c680fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:56.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.146+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 shutdown_connections 2023-06-22T18:03:56.529 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:56.146+0000 7f4c6fc63700 1 -- 172.21.15.39:0/2113016012 wait complete. 2023-06-22T18:03:56.529 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2023-06-22T18:03:56.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:56 smithi039 ceph-mon[109976]: mgrmap e12: x(active, since 2s) 2023-06-22T18:03:56.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:56 smithi039 ceph-mon[109976]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:56.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:03:56.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:03:56.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:03:58.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$0H0bwayqLwIBqb20wNsm.eQWzIs6vQPKjowssqRVSKw7cnaPuaWFW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1687457037, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-06-22T18:03:58.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.091+0000 7f61b7e4e700 1 Processor -- start 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.091+0000 7f61b7e4e700 1 -- start start 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.091+0000 7f61b7e4e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b01025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.091+0000 7f61b7e4e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0102bc0 con 0x7f61b01021e0 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.092+0000 7f61b5bea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b01025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.092+0000 7f61b5bea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b01025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49570/0 (socket says 172.21.15.39:49570) 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.092+0000 7f61b5bea700 1 -- 172.21.15.39:0/3636683499 learned_addr learned my addr 172.21.15.39:0/3636683499 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:58.006 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.092+0000 7f61b5bea700 1 -- 172.21.15.39:0/3636683499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b01033f0 con 0x7f61b01021e0 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.092+0000 7f61b5bea700 1 --2- 172.21.15.39:0/3636683499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b01025f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098e0 tx=0x7f61ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b706ecc5732fa00 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b4be8700 1 -- 172.21.15.39:0/3636683499 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61ac009e80 con 0x7f61b01021e0 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b4be8700 1 -- 172.21.15.39:0/3636683499 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ac00bd30 con 0x7f61b01021e0 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b4be8700 1 -- 172.21.15.39:0/3636683499 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61ac011ca0 con 0x7f61b01021e0 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3636683499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 msgr2=0x7f61b01025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b7e4e700 1 --2- 172.21.15.39:0/3636683499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b01025f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0098e0 tx=0x7f61ac009250 comp rx=0 tx=0).stop 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3636683499 shutdown_connections 2023-06-22T18:03:58.007 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b7e4e700 1 --2- 172.21.15.39:0/3636683499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b01025f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.093+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3636683499 >> 172.21.15.39:0/3636683499 conn(0x7f61b00fd8a0 msgr2=0x7f61b00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.094+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3636683499 shutdown_connections 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.094+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3636683499 wait complete. 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.094+0000 7f61b7e4e700 1 Processor -- start 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.094+0000 7f61b7e4e700 1 -- start start 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.095+0000 7f61b7e4e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b0113120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.095+0000 7f61b7e4e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0113660 con 0x7f61b01021e0 2023-06-22T18:03:58.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.095+0000 7f61b5bea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b0113120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:58.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.095+0000 7f61b5bea700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b0113120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49582/0 (socket says 172.21.15.39:49582) 2023-06-22T18:03:58.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.095+0000 7f61b5bea700 1 -- 172.21.15.39:0/3541578429 learned_addr learned my addr 172.21.15.39:0/3541578429 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:58.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.095+0000 7f61b5bea700 1 -- 172.21.15.39:0/3541578429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0094f0 con 0x7f61b01021e0 2023-06-22T18:03:58.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.096+0000 7f61b5bea700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b0113120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006fd0 tx=0x7f61ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:58.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.096+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61ac01a710 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.096+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ac01ad40 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.096+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b0113860 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.096+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61ac022c90 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.096+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b0113c70 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.097+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b005f060 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.098+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f61ac022500 con 0x7f61b01021e0 2023-06-22T18:03:58.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.098+0000 7f61a67fc700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f619c036610 0x7f619c038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:58.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.098+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f61ac04a730 con 0x7f61b01021e0 2023-06-22T18:03:58.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.098+0000 7f61b53e9700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f619c036610 0x7f619c038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:58.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.099+0000 7f61b53e9700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f619c036610 0x7f619c038ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009990 tx=0x7f61a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:58.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.103+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f61ac0219f0 con 0x7f61b01021e0 2023-06-22T18:03:58.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.327+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f61b005ee80 con 0x7f619c036610 2023-06-22T18:03:58.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.579+0000 7f61a67fc700 1 -- 172.21.15.39:0/3541578429 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f61b005ee80 con 0x7f619c036610 2023-06-22T18:03:58.012 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.581+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f619c036610 msgr2=0x7f619c038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:58.012 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f619c036610 0x7f619c038ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009990 tx=0x7f61a0006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:58.012 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 msgr2=0x7f61b0113120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:58.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b0113120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006fd0 tx=0x7f61ac009250 comp rx=0 tx=0).stop 2023-06-22T18:03:58.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 shutdown_connections 2023-06-22T18:03:58.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f619c036610 0x7f619c038ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:58.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 --2- 172.21.15.39:0/3541578429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61b01021e0 0x7f61b0113120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:58.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 >> 172.21.15.39:0/3541578429 conn(0x7f61b00fd8a0 msgr2=0x7f61b00fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:58.013 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 shutdown_connections 2023-06-22T18:03:58.014 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:57.582+0000 7f61b7e4e700 1 -- 172.21.15.39:0/3541578429 wait complete. 2023-06-22T18:03:58.014 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2023-06-22T18:03:58.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:58 smithi039 ceph-mon[109976]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:03:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:03:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:03:59.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2023-06-22T18:03:59.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.621+0000 7f780f5a8700 1 Processor -- start 2023-06-22T18:03:59.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.621+0000 7f780f5a8700 1 -- start start 2023-06-22T18:03:59.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.622+0000 7f780f5a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:59.490 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.622+0000 7f780f5a8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808076640 con 0x7f7808077c90 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.622+0000 7f780d344700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.622+0000 7f780d344700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49596/0 (socket says 172.21.15.39:49596) 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.622+0000 7f780d344700 1 -- 172.21.15.39:0/3155537606 learned_addr learned my addr 172.21.15.39:0/3155537606 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.622+0000 7f780d344700 1 -- 172.21.15.39:0/3155537606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808076780 con 0x7f7808077c90 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.623+0000 7f780d344700 1 --2- 172.21.15.39:0/3155537606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808076100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78040098e0 tx=0x7f7804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75313c681261ab5 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.623+0000 7f77fffff700 1 -- 172.21.15.39:0/3155537606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7804009e80 con 0x7f7808077c90 2023-06-22T18:03:59.491 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.623+0000 7f77fffff700 1 -- 172.21.15.39:0/3155537606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780400bd30 con 0x7f7808077c90 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.623+0000 7f77fffff700 1 -- 172.21.15.39:0/3155537606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7804011ca0 con 0x7f7808077c90 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 -- 172.21.15.39:0/3155537606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 msgr2=0x7f7808076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 --2- 172.21.15.39:0/3155537606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808076100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f78040098e0 tx=0x7f7804009250 comp rx=0 tx=0).stop 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 -- 172.21.15.39:0/3155537606 shutdown_connections 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 --2- 172.21.15.39:0/3155537606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808076100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 -- 172.21.15.39:0/3155537606 >> 172.21.15.39:0/3155537606 conn(0x7f78080fd8a0 msgr2=0x7f78080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:59.492 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 -- 172.21.15.39:0/3155537606 shutdown_connections 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.624+0000 7f780f5a8700 1 -- 172.21.15.39:0/3155537606 wait complete. 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.625+0000 7f780f5a8700 1 Processor -- start 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.625+0000 7f780f5a8700 1 -- start start 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780f5a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780f5a8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78081135d0 con 0x7f7808077c90 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780d344700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:59.493 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780d344700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808113090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49598/0 (socket says 172.21.15.39:49598) 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780d344700 1 -- 172.21.15.39:0/1984533867 learned_addr learned my addr 172.21.15.39:0/1984533867 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780d344700 1 -- 172.21.15.39:0/1984533867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78040094f0 con 0x7f7808077c90 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.626+0000 7f780d344700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808113090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c00 tx=0x7f7804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.627+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780401a760 con 0x7f7808077c90 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.627+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f780401ad90 con 0x7f7808077c90 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.627+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7804022c60 con 0x7f7808077c90 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.627+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78081137d0 con 0x7f7808077c90 2023-06-22T18:03:59.494 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.627+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7808113be0 con 0x7f7808077c90 2023-06-22T18:03:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.628+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780804b970 con 0x7f7808077c90 2023-06-22T18:03:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.628+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7f7804022570 con 0x7f7808077c90 2023-06-22T18:03:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.629+0000 7f77fdffb700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f77f40365c0 0x7f77f4038a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:03:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.629+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f780404a710 con 0x7f7808077c90 2023-06-22T18:03:59.495 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.629+0000 7f780cb43700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f77f40365c0 0x7f77f4038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:03:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.630+0000 7f780cb43700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f77f40365c0 0x7f77f4038a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:03:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.633+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f780402ab80 con 0x7f7808077c90 2023-06-22T18:03:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.850+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f78081085d0 con 0x7f7808077c90 2023-06-22T18:03:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.851+0000 7f77fdffb700 1 -- 172.21.15.39:0/1984533867 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f780402ab80 con 0x7f7808077c90 2023-06-22T18:03:59.496 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f77f40365c0 msgr2=0x7f77f4038a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f77f40365c0 0x7f77f4038a70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009990 tx=0x7f77f8006e40 comp rx=0 tx=0).stop 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 msgr2=0x7f7808113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808113090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7804000c00 tx=0x7f7804009250 comp rx=0 tx=0).stop 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 shutdown_connections 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f77f40365c0 0x7f77f4038a70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 --2- 172.21.15.39:0/1984533867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808077c90 0x7f7808113090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:03:59.497 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 >> 172.21.15.39:0/1984533867 conn(0x7f78080fd8a0 msgr2=0x7f78080fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:03:59.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 shutdown_connections 2023-06-22T18:03:59.498 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:03:58.854+0000 7f780f5a8700 1 -- 172.21.15.39:0/1984533867 wait complete. 2023-06-22T18:03:59.509 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-22T18:03:59.510 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2023-06-22T18:03:59.528 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-22T18:03:59.528 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2023-06-22T18:03:59.528 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2023-06-22T18:03:59.528 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-06-22T18:03:59.530 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2023-06-22T18:03:59.530 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:03:59.531 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039.front.sepia.ceph.com:8443/ 2023-06-22T18:03:59.531 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2023-06-22T18:03:59.531 INFO:teuthology.orchestra.run.smithi039.stdout: Password: r33n8lqd31 2023-06-22T18:03:59.531 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:03:59.531 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config directory 2023-06-22T18:03:59.532 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling autotune for osd_memory_target 2023-06-22T18:03:59.747 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:03:59 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1984533867' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-06-22T18:04:00.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.118+0000 7ff6bae68700 1 Processor -- start 2023-06-22T18:04:00.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.119+0000 7ff6bae68700 1 -- start start 2023-06-22T18:04:00.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.119+0000 7ff6bae68700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:00.758 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.119+0000 7ff6bae68700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4102bc0 con 0x7ff6b41021e0 2023-06-22T18:04:00.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.120+0000 7ff6b8c04700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:00.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.120+0000 7ff6b8c04700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49600/0 (socket says 172.21.15.39:49600) 2023-06-22T18:04:00.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.120+0000 7ff6b8c04700 1 -- 172.21.15.39:0/4083428228 learned_addr learned my addr 172.21.15.39:0/4083428228 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:00.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.120+0000 7ff6b8c04700 1 -- 172.21.15.39:0/4083428228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b41033f0 con 0x7ff6b41021e0 2023-06-22T18:04:00.759 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.120+0000 7ff6b8c04700 1 --2- 172.21.15.39:0/4083428228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41025f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80098e0 tx=0x7ff6a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9349976ebbc5f5b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:00.760 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6b37fe700 1 -- 172.21.15.39:0/4083428228 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6a8009e80 con 0x7ff6b41021e0 2023-06-22T18:04:00.760 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6b37fe700 1 -- 172.21.15.39:0/4083428228 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6a800bd30 con 0x7ff6b41021e0 2023-06-22T18:04:00.760 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6b37fe700 1 -- 172.21.15.39:0/4083428228 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6a8011ca0 con 0x7ff6b41021e0 2023-06-22T18:04:00.760 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 -- 172.21.15.39:0/4083428228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 msgr2=0x7ff6b41025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 --2- 172.21.15.39:0/4083428228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41025f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80098e0 tx=0x7ff6a8009250 comp rx=0 tx=0).stop 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 -- 172.21.15.39:0/4083428228 shutdown_connections 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 --2- 172.21.15.39:0/4083428228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41025f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 -- 172.21.15.39:0/4083428228 >> 172.21.15.39:0/4083428228 conn(0x7ff6b40fd8a0 msgr2=0x7ff6b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 -- 172.21.15.39:0/4083428228 shutdown_connections 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.121+0000 7ff6bae68700 1 -- 172.21.15.39:0/4083428228 wait complete. 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.122+0000 7ff6bae68700 1 Processor -- start 2023-06-22T18:04:00.761 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.122+0000 7ff6bae68700 1 -- start start 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.123+0000 7ff6bae68700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41130f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.123+0000 7ff6bae68700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4113630 con 0x7ff6b41021e0 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.123+0000 7ff6b8c04700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.123+0000 7ff6b8c04700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49610/0 (socket says 172.21.15.39:49610) 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.123+0000 7ff6b8c04700 1 -- 172.21.15.39:0/1168561321 learned_addr learned my addr 172.21.15.39:0/1168561321 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.123+0000 7ff6b8c04700 1 -- 172.21.15.39:0/1168561321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80094f0 con 0x7ff6b41021e0 2023-06-22T18:04:00.762 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.124+0000 7ff6b8c04700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41130f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8006fd0 tx=0x7ff6a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.124+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6a801a710 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.124+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6a801ad40 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.124+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b4113830 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.124+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6a8022c90 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.124+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b4113c40 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.125+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 12) v1 ==== 43232+0+0 (secure 0 0 0) 0x7ff6a801a870 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.125+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b405f060 con 0x7ff6b41021e0 2023-06-22T18:04:00.763 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.125+0000 7ff6b17fa700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff6a4036610 0x7ff6a4038ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.125+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff6a804a700 con 0x7ff6b41021e0 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.126+0000 7ff6b3fff700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff6a4036610 0x7ff6a4038ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.126+0000 7ff6b3fff700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff6a4036610 0x7ff6a4038ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009990 tx=0x7ff6a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.131+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff6a80219f0 con 0x7ff6b41021e0 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.334+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7ff6b404b970 con 0x7ff6b41021e0 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.335+0000 7ff6b17fa700 1 -- 172.21.15.39:0/1168561321 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7ff6a80213a0 con 0x7ff6b41021e0 2023-06-22T18:04:00.764 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff6a4036610 msgr2=0x7ff6a4038ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff6a4036610 0x7ff6a4038ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009990 tx=0x7ff6a0006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 msgr2=0x7ff6b41130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41130f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8006fd0 tx=0x7ff6a8009250 comp rx=0 tx=0).stop 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 shutdown_connections 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff6a4036610 0x7ff6a4038ac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 --2- 172.21.15.39:0/1168561321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b41021e0 0x7ff6b41130f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 >> 172.21.15.39:0/1168561321 conn(0x7ff6b40fd8a0 msgr2=0x7ff6b40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:00.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.337+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 shutdown_connections 2023-06-22T18:04:00.766 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:00.338+0000 7ff6bae68700 1 -- 172.21.15.39:0/1168561321 wait complete. 2023-06-22T18:04:01.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:01 smithi039 ceph-mon[109976]: mgrmap e13: x(active, since 6s) 2023-06-22T18:04:02.128 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.286+0000 7f21d491c700 1 Processor -- start 2023-06-22T18:04:02.128 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.286+0000 7f21d491c700 1 -- start start 2023-06-22T18:04:02.128 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.286+0000 7f21d491c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d00760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.286+0000 7f21d491c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0076620 con 0x7f21d0077c70 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.287+0000 7f21ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d00760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.287+0000 7f21ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d00760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49614/0 (socket says 172.21.15.39:49614) 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.287+0000 7f21ce59c700 1 -- 172.21.15.39:0/3938901142 learned_addr learned my addr 172.21.15.39:0/3938901142 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.287+0000 7f21ce59c700 1 -- 172.21.15.39:0/3938901142 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d0076760 con 0x7f21d0077c70 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.287+0000 7f21ce59c700 1 --2- 172.21.15.39:0/3938901142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d00760e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f21c00098e0 tx=0x7f21c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137c51903facdae9 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.287+0000 7f21cd59a700 1 -- 172.21.15.39:0/3938901142 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c0009e80 con 0x7f21d0077c70 2023-06-22T18:04:02.129 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21cd59a700 1 -- 172.21.15.39:0/3938901142 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21c000bd30 con 0x7f21d0077c70 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21cd59a700 1 -- 172.21.15.39:0/3938901142 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c0011ca0 con 0x7f21d0077c70 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 -- 172.21.15.39:0/3938901142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 msgr2=0x7f21d00760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 --2- 172.21.15.39:0/3938901142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d00760e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f21c00098e0 tx=0x7f21c0009250 comp rx=0 tx=0).stop 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 -- 172.21.15.39:0/3938901142 shutdown_connections 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 --2- 172.21.15.39:0/3938901142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d00760e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 -- 172.21.15.39:0/3938901142 >> 172.21.15.39:0/3938901142 conn(0x7f21d00fd8a0 msgr2=0x7f21d00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:02.130 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 -- 172.21.15.39:0/3938901142 shutdown_connections 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.288+0000 7f21d491c700 1 -- 172.21.15.39:0/3938901142 wait complete. 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.289+0000 7f21d491c700 1 Processor -- start 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.289+0000 7f21d491c700 1 -- start start 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21d491c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d0113090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21d491c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d01135d0 con 0x7f21d0077c70 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d0113090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21ce59c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d0113090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49620/0 (socket says 172.21.15.39:49620) 2023-06-22T18:04:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21ce59c700 1 -- 172.21.15.39:0/2493689144 learned_addr learned my addr 172.21.15.39:0/2493689144 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21ce59c700 1 -- 172.21.15.39:0/2493689144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00094f0 con 0x7f21d0077c70 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.290+0000 7f21ce59c700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d0113090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.291+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c001a790 con 0x7f21d0077c70 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.291+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21c001adc0 con 0x7f21d0077c70 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.291+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21c0019ce0 con 0x7f21d0077c70 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.291+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d01137d0 con 0x7f21d0077c70 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.291+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d0113be0 con 0x7f21d0077c70 2023-06-22T18:04:02.132 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.292+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d004b970 con 0x7f21d0077c70 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.292+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f21c00195c0 con 0x7f21d0077c70 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.292+0000 7f21beffd700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f21b80366b0 0x7f21b8038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.292+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f21c0049da0 con 0x7f21d0077c70 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.293+0000 7f21cdd9b700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f21b80366b0 0x7f21b8038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.293+0000 7f21cdd9b700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f21b80366b0 0x7f21b8038b60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009990 tx=0x7f21c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.297+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f21c002a8d0 con 0x7f21d0077c70 2023-06-22T18:04:02.133 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.610+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f21d0114710 con 0x7f21d0077c70 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.617+0000 7f21beffd700 1 -- 172.21.15.39:0/2493689144 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f21c001e020 con 0x7f21d0077c70 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.619+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f21b80366b0 msgr2=0x7f21b8038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.619+0000 7f21d491c700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f21b80366b0 0x7f21b8038b60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009990 tx=0x7f21c4006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.619+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 msgr2=0x7f21d0113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.619+0000 7f21d491c700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d0113090 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0009250 comp rx=0 tx=0).stop 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.620+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 shutdown_connections 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.620+0000 7f21d491c700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f21b80366b0 0x7f21b8038b60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:02.134 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.620+0000 7f21d491c700 1 --2- 172.21.15.39:0/2493689144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21d0077c70 0x7f21d0113090 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.620+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 >> 172.21.15.39:0/2493689144 conn(0x7f21d00fd8a0 msgr2=0x7f21d00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.620+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 shutdown_connections 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr 2023-06-22T18:04:01.620+0000 7f21d491c700 1 -- 172.21.15.39:0/2493689144 wait complete. 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:02.135 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2023-06-22T18:04:02.136 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.137 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2023-06-22T18:04:02.137 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.137 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2023-06-22T18:04:02.137 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:02.137 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2023-06-22T18:04:02.186 INFO:tasks.cephadm:Fetching config... 2023-06-22T18:04:02.187 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:02.187 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-06-22T18:04:02.223 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-06-22T18:04:02.224 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:02.224 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-06-22T18:04:02.301 INFO:tasks.cephadm:Fetching mon keyring... 2023-06-22T18:04:02.302 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:02.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-06-22T18:04:02.405 INFO:tasks.cephadm:Fetching pub ssh key... 2023-06-22T18:04:02.406 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:02.406 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-06-22T18:04:02.482 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-06-22T18:04:02.482 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-22T18:04:02.618 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:02.650 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:02 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2493689144' entity='client.admin' 2023-06-22T18:04:02.652 DEBUG:teuthology.orchestra.run.smithi101:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-22T18:04:02.692 INFO:teuthology.orchestra.run.smithi101.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:02.704 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-22T18:04:02.746 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGJCmmgXnUdxNiBKCSkwOhwUeODtmp2hrj9fuw203mOVJTz6tk4BGDVk619e2DA7YpLlphnyDBPipjsIR7kAK27rHxxOZDmtO+zBnFpi492YAqq46YTOsdm/wAsx8zUG7bvAaQpwtNYzmnK/vHY/KEpguu3YuyK1qaEiCtCU8ht6mExY0t8imbj1+XpA2YfUeuffwJ/0+Y2N7i/908ScTkTZEMb7+jbINph/Hq+KNutfls+EGAFiXa+rVkfanO33vRBbyodjhtBKo48EGGcj+/CwWFmwkQGoeeWtbVkmlWOsYI3OTHJgFbAoMhqB88Wr5mMAxbwr8HryLGVyD6NgAAmvH5gt2GBH/Q3+sZLpck9/mON9gXzYe0YKgCOIQYZMlTAzWQEIx8oskYjfVH7DgjKGbjotD6Rso67KxJ5yG1IYMK04AGtQjfKz8K9YtPVDTUUj+WpMwwEnaugCOTkeacDN7MkppryJhip8Y+r12as/FVhMLV+yLjQ97xW4CLBjc= ceph-d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:02.759 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-06-22T18:04:03.827 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:05.251 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 -- 172.21.15.39:0/2202864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 msgr2=0x7f76c006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 --2- 172.21.15.39:0/2202864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c006fdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76b00048c0 tx=0x7f76b0009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 -- 172.21.15.39:0/2202864666 shutdown_connections 2023-06-22T18:04:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 --2- 172.21.15.39:0/2202864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c006fdc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 -- 172.21.15.39:0/2202864666 >> 172.21.15.39:0/2202864666 conn(0x7f76c00f7dd0 msgr2=0x7f76c00fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 -- 172.21.15.39:0/2202864666 shutdown_connections 2023-06-22T18:04:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.247+0000 7f76c61e8700 1 -- 172.21.15.39:0/2202864666 wait complete. 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76c61e8700 1 Processor -- start 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76c61e8700 1 -- start start 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76c61e8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c010e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76c61e8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c010ef30 con 0x7f76c00718b0 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c010e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76bf7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c010e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49642/0 (socket says 172.21.15.39:49642) 2023-06-22T18:04:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.248+0000 7f76bf7fe700 1 -- 172.21.15.39:0/844660543 learned_addr learned my addr 172.21.15.39:0/844660543 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76bf7fe700 1 -- 172.21.15.39:0/844660543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0005040 con 0x7f76c00718b0 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76bf7fe700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c010e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000c00 tx=0x7f76b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76b000e040 con 0x7f76c00718b0 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c010f130 con 0x7f76c00718b0 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76b0009890 con 0x7f76c00718b0 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76b0013e90 con 0x7f76c00718b0 2023-06-22T18:04:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.249+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c010fdd0 con 0x7f76c00718b0 2023-06-22T18:04:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.250+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f76b0013620 con 0x7f76c00718b0 2023-06-22T18:04:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.250+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c005f060 con 0x7f76c00718b0 2023-06-22T18:04:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.251+0000 7f76a7fff700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f76a8036660 0x7f76a8038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.251+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f76b00495d0 con 0x7f76c00718b0 2023-06-22T18:04:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.251+0000 7f76beffd700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f76a8036660 0x7f76a8038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.251+0000 7f76beffd700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f76a8036660 0x7f76a8038b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009990 tx=0x7f76b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.256+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f76b0011960 con 0x7f76c00718b0 2023-06-22T18:04:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.457+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f76c004b970 con 0x7f76c00718b0 2023-06-22T18:04:05.465 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.461+0000 7f76a7fff700 1 -- 172.21.15.39:0/844660543 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f76b0011370 con 0x7f76c00718b0 2023-06-22T18:04:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.463+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f76a8036660 msgr2=0x7f76a8038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.463+0000 7f76c61e8700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f76a8036660 0x7f76a8038b10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009990 tx=0x7f76b4006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.463+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 msgr2=0x7f76c010e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c010e9f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f76b0000c00 tx=0x7f76b0009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:05.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 shutdown_connections 2023-06-22T18:04:05.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f76a8036660 0x7f76a8038b10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:05.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 --2- 172.21.15.39:0/844660543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c00718b0 0x7f76c010e9f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:05.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 >> 172.21.15.39:0/844660543 conn(0x7f76c00f7dd0 msgr2=0x7f76c00f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:05.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 shutdown_connections 2023-06-22T18:04:05.469 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:05.464+0000 7f76c61e8700 1 -- 172.21.15.39:0/844660543 wait complete. 2023-06-22T18:04:05.901 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-06-22T18:04:05.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-06-22T18:04:06.668 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:06.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:06 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/844660543' entity='client.admin' 2023-06-22T18:04:06.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:06.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 -- 172.21.15.39:0/733856763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 msgr2=0x7f6b980fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 --2- 172.21.15.39:0/733856763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b980fca70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b880048c0 tx=0x7f6b88009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 -- 172.21.15.39:0/733856763 shutdown_connections 2023-06-22T18:04:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 --2- 172.21.15.39:0/733856763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b980fca70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 -- 172.21.15.39:0/733856763 >> 172.21.15.39:0/733856763 conn(0x7f6b980f7d40 msgr2=0x7f6b980fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 -- 172.21.15.39:0/733856763 shutdown_connections 2023-06-22T18:04:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.174+0000 7f6b9e742700 1 -- 172.21.15.39:0/733856763 wait complete. 2023-06-22T18:04:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.175+0000 7f6b9e742700 1 Processor -- start 2023-06-22T18:04:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.175+0000 7f6b9e742700 1 -- start start 2023-06-22T18:04:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.175+0000 7f6b9e742700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b9810c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.175+0000 7f6b9e742700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9810cd60 con 0x7f6b980fc660 2023-06-22T18:04:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.176+0000 7f6b97fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b9810c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.176+0000 7f6b97fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b9810c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55840/0 (socket says 172.21.15.39:55840) 2023-06-22T18:04:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.176+0000 7f6b97fff700 1 -- 172.21.15.39:0/2209223701 learned_addr learned my addr 172.21.15.39:0/2209223701 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.176+0000 7f6b97fff700 1 -- 172.21.15.39:0/2209223701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b88005040 con 0x7f6b980fc660 2023-06-22T18:04:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.176+0000 7f6b97fff700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b9810c820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b88000c00 tx=0x7f6b88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.176+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b8800e040 con 0x7f6b980fc660 2023-06-22T18:04:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.177+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b88009890 con 0x7f6b980fc660 2023-06-22T18:04:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.177+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9810cf60 con 0x7f6b980fc660 2023-06-22T18:04:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.177+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b88013e90 con 0x7f6b980fc660 2023-06-22T18:04:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.177+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9810dc00 con 0x7f6b980fc660 2023-06-22T18:04:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.178+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9805f060 con 0x7f6b980fc660 2023-06-22T18:04:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.178+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f6b88013620 con 0x7f6b980fc660 2023-06-22T18:04:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.178+0000 7f6b957fa700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f6b840366b0 0x7f6b84038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.179+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6b88049570 con 0x7f6b980fc660 2023-06-22T18:04:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.179+0000 7f6b8ffff700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f6b840366b0 0x7f6b84038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.179+0000 7f6b8ffff700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f6b840366b0 0x7f6b84038b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009990 tx=0x7f6b80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.183+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f6b88011960 con 0x7f6b980fc660 2023-06-22T18:04:08.395 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.391+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b9810e740 con 0x7f6b840366b0 2023-06-22T18:04:08.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.395+0000 7f6b957fa700 1 -- 172.21.15.39:0/2209223701 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6b9810e740 con 0x7f6b840366b0 2023-06-22T18:04:08.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f6b840366b0 msgr2=0x7f6b84038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:08.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f6b840366b0 0x7f6b84038b60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009990 tx=0x7f6b80006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:08.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 msgr2=0x7f6b9810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:08.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b9810c820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b88000c00 tx=0x7f6b88009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:08.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 shutdown_connections 2023-06-22T18:04:08.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f6b840366b0 0x7f6b84038b60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:08.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 --2- 172.21.15.39:0/2209223701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b980fc660 0x7f6b9810c820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:08.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 >> 172.21.15.39:0/2209223701 conn(0x7f6b980f7d40 msgr2=0x7f6b980f8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:08.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 shutdown_connections 2023-06-22T18:04:08.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:08.398+0000 7f6b9e742700 1 -- 172.21.15.39:0/2209223701 wait complete. 2023-06-22T18:04:08.932 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi101 2023-06-22T18:04:08.932 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:04:08.932 DEBUG:teuthology.orchestra.run.smithi101:> dd of=/etc/ceph/ceph.conf 2023-06-22T18:04:08.949 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:04:08.950 DEBUG:teuthology.orchestra.run.smithi101:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:09.005 INFO:tasks.cephadm:Adding host smithi101 to orchestrator... 2023-06-22T18:04:09.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch host add smithi101 2023-06-22T18:04:09.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:09.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: Updating smithi039:/etc/ceph/ceph.conf 2023-06-22T18:04:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:09 smithi039 ceph-mon[109976]: Updating smithi039:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:09.809 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:10.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:10 smithi039 ceph-mon[109976]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:10.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:10 smithi039 ceph-mon[109976]: Updating smithi039:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:10.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:10.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:10.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:11.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.237+0000 7fe233a0b700 1 -- 172.21.15.39:0/1870091332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 msgr2=0x7fe22c102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:11.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.237+0000 7fe233a0b700 1 --2- 172.21.15.39:0/1870091332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c102a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe2280048c0 tx=0x7fe228009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:11.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.237+0000 7fe233a0b700 1 -- 172.21.15.39:0/1870091332 shutdown_connections 2023-06-22T18:04:11.242 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.237+0000 7fe233a0b700 1 --2- 172.21.15.39:0/1870091332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c102a90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:11.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.237+0000 7fe233a0b700 1 -- 172.21.15.39:0/1870091332 >> 172.21.15.39:0/1870091332 conn(0x7fe22c0f8200 msgr2=0x7fe22c0fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:11.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.237+0000 7fe233a0b700 1 -- 172.21.15.39:0/1870091332 shutdown_connections 2023-06-22T18:04:11.243 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.238+0000 7fe233a0b700 1 -- 172.21.15.39:0/1870091332 wait complete. 2023-06-22T18:04:11.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.238+0000 7fe233a0b700 1 Processor -- start 2023-06-22T18:04:11.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe233a0b700 1 -- start start 2023-06-22T18:04:11.244 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe233a0b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c1128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe233a0b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c112e20 con 0x7fe22c1026c0 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe2317a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c1128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe2317a7700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c1128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55860/0 (socket says 172.21.15.39:55860) 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe2317a7700 1 -- 172.21.15.39:0/4067217276 learned_addr learned my addr 172.21.15.39:0/4067217276 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.239+0000 7fe2317a7700 1 -- 172.21.15.39:0/4067217276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228005040 con 0x7fe22c1026c0 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.240+0000 7fe2317a7700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c1128e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe22800b600 tx=0x7fe228009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:11.245 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.240+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe22800e040 con 0x7fe22c1026c0 2023-06-22T18:04:11.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.240+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2280098c0 con 0x7fe22c1026c0 2023-06-22T18:04:11.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.240+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c113020 con 0x7fe22c1026c0 2023-06-22T18:04:11.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.240+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe228013de0 con 0x7fe22c1026c0 2023-06-22T18:04:11.246 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.240+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c113490 con 0x7fe22c1026c0 2023-06-22T18:04:11.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.241+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fe2280135b0 con 0x7fe22c1026c0 2023-06-22T18:04:11.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.241+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c05f060 con 0x7fe22c1026c0 2023-06-22T18:04:11.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.241+0000 7fe221ffb700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe2180366b0 0x7fe218038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:11.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.241+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe228049530 con 0x7fe22c1026c0 2023-06-22T18:04:11.247 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.242+0000 7fe230fa6700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe2180366b0 0x7fe218038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:11.248 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.242+0000 7fe230fa6700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe2180366b0 0x7fe218038b60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:11.250 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.247+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fe228023300 con 0x7fe22c1026c0 2023-06-22T18:04:11.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:11.456+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi101", "target": ["mon-mgr", ""]}) v1 -- 0x7fe22c04b970 con 0x7fe2180366b0 2023-06-22T18:04:12.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:12 smithi039 ceph-mon[109976]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi101", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:12.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:12 smithi039 ceph-mon[109976]: Deploying cephadm binary to smithi101 2023-06-22T18:04:13.249 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.245+0000 7fe221ffb700 1 -- 172.21.15.39:0/4067217276 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe22c04b970 con 0x7fe2180366b0 2023-06-22T18:04:13.250 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi101' with addr '172.21.15.101' 2023-06-22T18:04:13.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe2180366b0 msgr2=0x7fe218038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:13.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe2180366b0 0x7fe218038b60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:13.252 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 msgr2=0x7fe22c1128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c1128e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe22800b600 tx=0x7fe228009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 shutdown_connections 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe2180366b0 0x7fe218038b60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 --2- 172.21.15.39:0/4067217276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe22c1026c0 0x7fe22c1128e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 >> 172.21.15.39:0/4067217276 conn(0x7fe22c0f8200 msgr2=0x7fe22c0f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 shutdown_connections 2023-06-22T18:04:13.253 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:13.248+0000 7fe233a0b700 1 -- 172.21.15.39:0/4067217276 wait complete. 2023-06-22T18:04:13.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-22T18:04:14.455 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:14.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:14.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:14 smithi039 ceph-mon[109976]: Added host smithi101 2023-06-22T18:04:14.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:14.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:14 smithi039 ceph-mon[109976]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:14.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:15.635 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:15 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.862+0000 7fb6f2413700 1 -- 172.21.15.39:0/3420564902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 msgr2=0x7fb6ec104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.862+0000 7fb6f2413700 1 --2- 172.21.15.39:0/3420564902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec104c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0048c0 tx=0x7fb6dc009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.862+0000 7fb6f2413700 1 -- 172.21.15.39:0/3420564902 shutdown_connections 2023-06-22T18:04:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.862+0000 7fb6f2413700 1 --2- 172.21.15.39:0/3420564902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec104c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.862+0000 7fb6f2413700 1 -- 172.21.15.39:0/3420564902 >> 172.21.15.39:0/3420564902 conn(0x7fb6ec0fa3f0 msgr2=0x7fb6ec0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.863+0000 7fb6f2413700 1 -- 172.21.15.39:0/3420564902 shutdown_connections 2023-06-22T18:04:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.863+0000 7fb6f2413700 1 -- 172.21.15.39:0/3420564902 wait complete. 2023-06-22T18:04:15.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.863+0000 7fb6f2413700 1 Processor -- start 2023-06-22T18:04:15.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.863+0000 7fb6f2413700 1 -- start start 2023-06-22T18:04:15.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.864+0000 7fb6f2413700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec114ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:15.868 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.864+0000 7fb6f2413700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec115010 con 0x7fb6ec1048b0 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.864+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec114ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.864+0000 7fb6ebfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec114ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32964/0 (socket says 172.21.15.39:32964) 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.864+0000 7fb6ebfff700 1 -- 172.21.15.39:0/1235866112 learned_addr learned my addr 172.21.15.39:0/1235866112 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.864+0000 7fb6ebfff700 1 -- 172.21.15.39:0/1235866112 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc005040 con 0x7fb6ec1048b0 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.865+0000 7fb6ebfff700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec114ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00b600 tx=0x7fb6dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.865+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6dc00e040 con 0x7fb6ec1048b0 2023-06-22T18:04:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.865+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec115210 con 0x7fb6ec1048b0 2023-06-22T18:04:15.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.865+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6dc0098c0 con 0x7fb6ec1048b0 2023-06-22T18:04:15.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.865+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6dc013de0 con 0x7fb6ec1048b0 2023-06-22T18:04:15.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.865+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec10e990 con 0x7fb6ec1048b0 2023-06-22T18:04:15.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.866+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec05f060 con 0x7fb6ec1048b0 2023-06-22T18:04:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.866+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fb6dc0135b0 con 0x7fb6ec1048b0 2023-06-22T18:04:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.866+0000 7fb6e8ff9700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb6d4036660 0x7fb6d4038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.867+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb6dc0495d0 con 0x7fb6ec1048b0 2023-06-22T18:04:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.867+0000 7fb6eb7fe700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb6d4036660 0x7fb6d4038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.867+0000 7fb6eb7fe700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb6d4036660 0x7fb6d4038b10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009990 tx=0x7fb6e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:15.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:15.872+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fb6dc023300 con 0x7fb6ec1048b0 2023-06-22T18:04:16.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.080+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6ec04b9e0 con 0x7fb6d4036660 2023-06-22T18:04:16.084 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.081+0000 7fb6e8ff9700 1 -- 172.21.15.39:0/1235866112 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fb6ec04b9e0 con 0x7fb6d4036660 2023-06-22T18:04:16.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:16.085 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.101", "hostname": "smithi101", "labels": [], "status": ""}] 2023-06-22T18:04:16.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb6d4036660 msgr2=0x7fb6d4038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:16.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb6d4036660 0x7fb6d4038b10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009990 tx=0x7fb6e0006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:16.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 msgr2=0x7fb6ec114ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:16.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec114ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00b600 tx=0x7fb6dc009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:16.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 shutdown_connections 2023-06-22T18:04:16.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb6d4036660 0x7fb6d4038b10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:16.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 --2- 172.21.15.39:0/1235866112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ec1048b0 0x7fb6ec114ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:16.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 >> 172.21.15.39:0/1235866112 conn(0x7fb6ec0fa3f0 msgr2=0x7fb6ec0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:16.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 shutdown_connections 2023-06-22T18:04:16.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:16.084+0000 7fb6f2413700 1 -- 172.21.15.39:0/1235866112 wait complete. 2023-06-22T18:04:16.516 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2023-06-22T18:04:16.517 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:04:16.517 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2023-06-22T18:04:16.534 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:04:16.534 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:16.590 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2023-06-22T18:04:16.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch host add smithi133 2023-06-22T18:04:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:16 smithi039 ceph-mon[109976]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:17.937 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:17 smithi039 ceph-mon[109976]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-22T18:04:18.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/2853109934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 msgr2=0x7f3e78070120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 --2- 172.21.15.39:0/2853109934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78070120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e680048c0 tx=0x7f3e68009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/2853109934 shutdown_connections 2023-06-22T18:04:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 --2- 172.21.15.39:0/2853109934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78070120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/2853109934 >> 172.21.15.39:0/2853109934 conn(0x7f3e780f8200 msgr2=0x7f3e780fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/2853109934 shutdown_connections 2023-06-22T18:04:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.772+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/2853109934 wait complete. 2023-06-22T18:04:18.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e7e3d3700 1 Processor -- start 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e7e3d3700 1 -- start start 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e7e3d3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78114b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e7e3d3700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78115060 con 0x7f3e7806fd50 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78114b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e77fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78114b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32986/0 (socket says 172.21.15.39:32986) 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.773+0000 7f3e77fff700 1 -- 172.21.15.39:0/1869719794 learned_addr learned my addr 172.21.15.39:0/1869719794 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:18.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e77fff700 1 -- 172.21.15.39:0/1869719794 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68005040 con 0x7f3e7806fd50 2023-06-22T18:04:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e77fff700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78114b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800b600 tx=0x7f3e68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e6800e040 con 0x7f3e7806fd50 2023-06-22T18:04:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e680098c0 con 0x7f3e7806fd50 2023-06-22T18:04:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e68013de0 con 0x7f3e7806fd50 2023-06-22T18:04:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e78115260 con 0x7f3e7806fd50 2023-06-22T18:04:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.774+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781156d0 con 0x7f3e7806fd50 2023-06-22T18:04:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.775+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7805f060 con 0x7f3e7806fd50 2023-06-22T18:04:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.776+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f3e680135b0 con 0x7f3e7806fd50 2023-06-22T18:04:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.776+0000 7f3e74ff9700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e600366b0 0x7f3e60038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.776+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3e68049540 con 0x7f3e7806fd50 2023-06-22T18:04:18.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.776+0000 7f3e777fe700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e600366b0 0x7f3e60038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:18.781 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.777+0000 7f3e777fe700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e600366b0 0x7f3e60038b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009990 tx=0x7f3e6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:18.784 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.781+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f3e68023300 con 0x7f3e7806fd50 2023-06-22T18:04:18.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:18 smithi039 ceph-mon[109976]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:18.996 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:18.993+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e7804b9e0 con 0x7f3e600366b0 2023-06-22T18:04:20.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:20.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:20.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:20.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: Updating smithi101:/etc/ceph/ceph.conf 2023-06-22T18:04:20.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:20.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.198 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:20.838 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.835+0000 7f3e74ff9700 1 -- 172.21.15.39:0/1869719794 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3e7804b9e0 con 0x7f3e600366b0 2023-06-22T18:04:20.838 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi133' with addr '172.21.15.133' 2023-06-22T18:04:20.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.837+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e600366b0 msgr2=0x7f3e60038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:20.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.837+0000 7f3e7e3d3700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e600366b0 0x7f3e60038b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009990 tx=0x7f3e6c006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:20.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.837+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 msgr2=0x7f3e78114b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:20.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.837+0000 7f3e7e3d3700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78114b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800b600 tx=0x7f3e68009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:20.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.838+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 shutdown_connections 2023-06-22T18:04:20.841 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.838+0000 7f3e7e3d3700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e600366b0 0x7f3e60038b60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:20.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.838+0000 7f3e7e3d3700 1 --2- 172.21.15.39:0/1869719794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e7806fd50 0x7f3e78114b20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:20.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.838+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 >> 172.21.15.39:0/1869719794 conn(0x7f3e780f8200 msgr2=0x7f3e78103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:20.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.838+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 shutdown_connections 2023-06-22T18:04:20.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:20.838+0000 7f3e7e3d3700 1 -- 172.21.15.39:0/1869719794 wait complete. 2023-06-22T18:04:21.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: Deploying cephadm binary to smithi133 2023-06-22T18:04:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:21.325 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-22T18:04:22.087 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:22.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:21 smithi039 ceph-mon[109976]: Added host smithi133 2023-06-22T18:04:22.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:22 smithi039 ceph-mon[109976]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:23.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 -- 172.21.15.39:0/1939303702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 msgr2=0x7ff0400fb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:23.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 --2- 172.21.15.39:0/1939303702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff0400fb840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0300048c0 tx=0x7ff030009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:23.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 -- 172.21.15.39:0/1939303702 shutdown_connections 2023-06-22T18:04:23.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 --2- 172.21.15.39:0/1939303702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff0400fb840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:23.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 -- 172.21.15.39:0/1939303702 >> 172.21.15.39:0/1939303702 conn(0x7ff0400713d0 msgr2=0x7ff0400717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:23.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 -- 172.21.15.39:0/1939303702 shutdown_connections 2023-06-22T18:04:23.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.610+0000 7ff046859700 1 -- 172.21.15.39:0/1939303702 wait complete. 2023-06-22T18:04:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff046859700 1 Processor -- start 2023-06-22T18:04:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff046859700 1 -- start start 2023-06-22T18:04:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff046859700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff040114af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff046859700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040115030 con 0x7ff0400fb470 2023-06-22T18:04:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff040114af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff040114af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32998/0 (socket says 172.21.15.39:32998) 2023-06-22T18:04:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.611+0000 7ff03ffff700 1 -- 172.21.15.39:0/3925869372 learned_addr learned my addr 172.21.15.39:0/3925869372 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff03ffff700 1 -- 172.21.15.39:0/3925869372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030005040 con 0x7ff0400fb470 2023-06-22T18:04:23.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff03ffff700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff040114af0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff030005010 tx=0x7ff030009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:23.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff03000e040 con 0x7ff0400fb470 2023-06-22T18:04:23.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff030009890 con 0x7ff0400fb470 2023-06-22T18:04:23.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff030013e20 con 0x7ff0400fb470 2023-06-22T18:04:23.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff040115230 con 0x7ff0400fb470 2023-06-22T18:04:23.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.612+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0401156a0 con 0x7ff0400fb470 2023-06-22T18:04:23.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.613+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7ff0300093c0 con 0x7ff0400fb470 2023-06-22T18:04:23.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.613+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04005f060 con 0x7ff0400fb470 2023-06-22T18:04:23.618 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.614+0000 7ff03cff9700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff0280366b0 0x7ff028038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:23.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.614+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff030049550 con 0x7ff0400fb470 2023-06-22T18:04:23.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.614+0000 7ff03f7fe700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff0280366b0 0x7ff028038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:23.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.615+0000 7ff03f7fe700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff0280366b0 0x7ff028038b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff034009990 tx=0x7ff034006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:23.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.619+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff030011960 con 0x7ff0400fb470 2023-06-22T18:04:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.829+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff04004b970 con 0x7ff0280366b0 2023-06-22T18:04:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.830+0000 7ff03cff9700 1 -- 172.21.15.39:0/3925869372 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7ff04004b970 con 0x7ff0280366b0 2023-06-22T18:04:23.833 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:04:23.833 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.101", "hostname": "smithi101", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2023-06-22T18:04:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff0280366b0 msgr2=0x7ff028038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff0280366b0 0x7ff028038b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff034009990 tx=0x7ff034006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 msgr2=0x7ff040114af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff040114af0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff030005010 tx=0x7ff030009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 shutdown_connections 2023-06-22T18:04:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff0280366b0 0x7ff028038b60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 --2- 172.21.15.39:0/3925869372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0400fb470 0x7ff040114af0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 >> 172.21.15.39:0/3925869372 conn(0x7ff0400713d0 msgr2=0x7ff0400fa230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 shutdown_connections 2023-06-22T18:04:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:23.832+0000 7ff046859700 1 -- 172.21.15.39:0/3925869372 wait complete. 2023-06-22T18:04:24.349 INFO:tasks.cephadm:Setting crush tunables to default 2023-06-22T18:04:24.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph osd crush tunables default 2023-06-22T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:24 smithi039 ceph-mon[109976]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:24.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:24 smithi039 ceph-mon[109976]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-22T18:04:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:26.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 -- 172.21.15.39:0/272883948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 msgr2=0x7fa974102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:26.514 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 --2- 172.21.15.39:0/272883948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974102a90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9700048c0 tx=0x7fa970009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 -- 172.21.15.39:0/272883948 shutdown_connections 2023-06-22T18:04:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 --2- 172.21.15.39:0/272883948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974102a90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 -- 172.21.15.39:0/272883948 >> 172.21.15.39:0/272883948 conn(0x7fa9740f8200 msgr2=0x7fa9740fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 -- 172.21.15.39:0/272883948 shutdown_connections 2023-06-22T18:04:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.510+0000 7fa97c508700 1 -- 172.21.15.39:0/272883948 wait complete. 2023-06-22T18:04:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.511+0000 7fa97c508700 1 Processor -- start 2023-06-22T18:04:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.511+0000 7fa97c508700 1 -- start start 2023-06-22T18:04:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97c508700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974114b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97c508700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974115080 con 0x7fa9741026c0 2023-06-22T18:04:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97a2a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974114b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97a2a4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974114b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40734/0 (socket says 172.21.15.39:40734) 2023-06-22T18:04:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97a2a4700 1 -- 172.21.15.39:0/1872430045 learned_addr learned my addr 172.21.15.39:0/1872430045 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97a2a4700 1 -- 172.21.15.39:0/1872430045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970005040 con 0x7fa9741026c0 2023-06-22T18:04:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.512+0000 7fa97a2a4700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974114b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa97000b600 tx=0x7fa970009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.513+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa97000e040 con 0x7fa9741026c0 2023-06-22T18:04:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.513+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9700098c0 con 0x7fa9741026c0 2023-06-22T18:04:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.513+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa970013de0 con 0x7fa9741026c0 2023-06-22T18:04:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.513+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa974115280 con 0x7fa9741026c0 2023-06-22T18:04:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.513+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9741156f0 con 0x7fa9741026c0 2023-06-22T18:04:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.514+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fa9700135b0 con 0x7fa9741026c0 2023-06-22T18:04:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.514+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97405f060 con 0x7fa9741026c0 2023-06-22T18:04:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.514+0000 7fa96affd700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa9600366b0 0x7fa960038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.514+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa970049540 con 0x7fa9741026c0 2023-06-22T18:04:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.515+0000 7fa979aa3700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa9600366b0 0x7fa960038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:26.520 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.515+0000 7fa979aa3700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa9600366b0 0x7fa960038b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa964009990 tx=0x7fa964006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:26.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.520+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fa970023300 con 0x7fa9741026c0 2023-06-22T18:04:26.731 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:26.728+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa97404b970 con 0x7fa9741026c0 2023-06-22T18:04:26.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:26.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:26.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:26.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:26.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:26.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:26.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:26.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:27.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.670+0000 7fa96affd700 1 -- 172.21.15.39:0/1872430045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa970011b60 con 0x7fa9741026c0 2023-06-22T18:04:27.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa9600366b0 msgr2=0x7fa960038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:27.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa9600366b0 0x7fa960038b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa964009990 tx=0x7fa964006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:27.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 msgr2=0x7fa974114b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:27.676 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974114b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa97000b600 tx=0x7fa970009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:27.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 shutdown_connections 2023-06-22T18:04:27.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa9600366b0 0x7fa960038b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:27.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 --2- 172.21.15.39:0/1872430045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9741026c0 0x7fa974114b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:27.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.672+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 >> 172.21.15.39:0/1872430045 conn(0x7fa9740f8200 msgr2=0x7fa97406fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:27.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.673+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 shutdown_connections 2023-06-22T18:04:27.677 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:27.673+0000 7fa97c508700 1 -- 172.21.15.39:0/1872430045 wait complete. 2023-06-22T18:04:27.678 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2023-06-22T18:04:27.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: Updating smithi133:/etc/ceph/ceph.conf 2023-06-22T18:04:27.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:27.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1872430045' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-22T18:04:27.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:27.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:27.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:27.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:27 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:28.082 INFO:tasks.cephadm:Adding mon.a on smithi039 2023-06-22T18:04:28.082 INFO:tasks.cephadm:Adding mon.b on smithi101 2023-06-22T18:04:28.083 INFO:tasks.cephadm:Adding mon.c on smithi133 2023-06-22T18:04:28.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c' 2023-06-22T18:04:28.693 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:28.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:28 smithi039 ceph-mon[109976]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:28.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:28 smithi039 ceph-mon[109976]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:28.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:28 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1872430045' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-22T18:04:28.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:28 smithi039 ceph-mon[109976]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:30.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 -- 172.21.15.133:0/2921062790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 msgr2=0x7fc770101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:30.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 --2- 172.21.15.133:0/2921062790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770101980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc7600048c0 tx=0x7fc760009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:30.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 -- 172.21.15.133:0/2921062790 shutdown_connections 2023-06-22T18:04:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 --2- 172.21.15.133:0/2921062790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770101980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 -- 172.21.15.133:0/2921062790 >> 172.21.15.133:0/2921062790 conn(0x7fc7700713d0 msgr2=0x7fc7700717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 -- 172.21.15.133:0/2921062790 shutdown_connections 2023-06-22T18:04:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.276+0000 7fc776832700 1 -- 172.21.15.133:0/2921062790 wait complete. 2023-06-22T18:04:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc776832700 1 Processor -- start 2023-06-22T18:04:30.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc776832700 1 -- start start 2023-06-22T18:04:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc776832700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc776832700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc770112db0 con 0x7fc7701015b0 2023-06-22T18:04:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc76ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc76ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770112870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.133:45248/0 (socket says 172.21.15.133:45248) 2023-06-22T18:04:30.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.277+0000 7fc76ffff700 1 -- 172.21.15.133:0/2953452415 learned_addr learned my addr 172.21.15.133:0/2953452415 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc76ffff700 1 -- 172.21.15.133:0/2953452415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760005040 con 0x7fc7701015b0 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc76ffff700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770112870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c010 tx=0x7fc760009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc76000e040 con 0x7fc7701015b0 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc770112fb0 con 0x7fc7701015b0 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc760009910 con 0x7fc7701015b0 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc760013e60 con 0x7fc7701015b0 2023-06-22T18:04:30.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.278+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77010c730 con 0x7fc7701015b0 2023-06-22T18:04:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.279+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fc760013640 con 0x7fc7701015b0 2023-06-22T18:04:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.279+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77005f060 con 0x7fc7701015b0 2023-06-22T18:04:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.279+0000 7fc76cff9700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc758036660 0x7fc758038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.279+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc760049500 con 0x7fc7701015b0 2023-06-22T18:04:30.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.280+0000 7fc76f7fe700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc758036660 0x7fc758038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:30.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.280+0000 7fc76f7fe700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc758036660 0x7fc758038b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc764009990 tx=0x7fc764006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:30.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.285+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc760011550 con 0x7fc7701015b0 2023-06-22T18:04:30.482 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.478+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fc77010de50 con 0x7fc758036660 2023-06-22T18:04:30.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.490+0000 7fc76cff9700 1 -- 172.21.15.133:0/2953452415 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc77010de50 con 0x7fc758036660 2023-06-22T18:04:30.494 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mon update... 2023-06-22T18:04:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.492+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc758036660 msgr2=0x7fc758038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.492+0000 7fc776832700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc758036660 0x7fc758038b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc764009990 tx=0x7fc764006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.492+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 msgr2=0x7fc770112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:30.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.492+0000 7fc776832700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770112870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c010 tx=0x7fc760009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.493+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 shutdown_connections 2023-06-22T18:04:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.493+0000 7fc776832700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc758036660 0x7fc758038b10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.493+0000 7fc776832700 1 --2- 172.21.15.133:0/2953452415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7701015b0 0x7fc770112870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.493+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 >> 172.21.15.133:0/2953452415 conn(0x7fc7700713d0 msgr2=0x7fc770102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.493+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 shutdown_connections 2023-06-22T18:04:30.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:30.493+0000 7fc776832700 1 -- 172.21.15.133:0/2953452415 wait complete. 2023-06-22T18:04:30.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:30.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:31.250 DEBUG:teuthology.orchestra.run.smithi101:mon.b> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.b.service 2023-06-22T18:04:31.253 DEBUG:teuthology.orchestra.run.smithi133:mon.c> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.c.service 2023-06-22T18:04:31.254 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-22T18:04:31.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-22T18:04:31.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:04:31.285 INFO:journalctl@ceph.mon.b.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:04:31.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:31 smithi039 ceph-mon[109976]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:31.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:31 smithi039 ceph-mon[109976]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c;count:3 2023-06-22T18:04:31.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:31 smithi039 ceph-mon[109976]: Deploying daemon mon.c on smithi133 2023-06-22T18:04:32.507 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:32 smithi039 ceph-mon[109976]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 -- 172.21.15.133:0/4243498935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 msgr2=0x7f5b14101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 --2- 172.21.15.133:0/4243498935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b14101980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5b040048c0 tx=0x7f5b04009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 -- 172.21.15.133:0/4243498935 shutdown_connections 2023-06-22T18:04:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 --2- 172.21.15.133:0/4243498935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b14101980 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 -- 172.21.15.133:0/4243498935 >> 172.21.15.133:0/4243498935 conn(0x7f5b140713d0 msgr2=0x7f5b140717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 -- 172.21.15.133:0/4243498935 shutdown_connections 2023-06-22T18:04:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 -- 172.21.15.133:0/4243498935 wait complete. 2023-06-22T18:04:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 Processor -- start 2023-06-22T18:04:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.723+0000 7f5b1b56c700 1 -- start start 2023-06-22T18:04:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.724+0000 7f5b1b56c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b141116d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.724+0000 7f5b1b56c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14111c10 con 0x7f5b141015b0 2023-06-22T18:04:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.724+0000 7f5b19308700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b141116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.724+0000 7f5b19308700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b141116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.133:45254/0 (socket says 172.21.15.133:45254) 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.724+0000 7f5b19308700 1 -- 172.21.15.133:0/1462169073 learned_addr learned my addr 172.21.15.133:0/1462169073 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b19308700 1 -- 172.21.15.133:0/1462169073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04005040 con 0x7f5b141015b0 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b19308700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b141116d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c010 tx=0x7f5b04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0400e040 con 0x7f5b141015b0 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b04009910 con 0x7f5b141015b0 2023-06-22T18:04:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14111e10 con 0x7f5b141015b0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b04013e70 con 0x7f5b141015b0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.725+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b14112300 con 0x7f5b141015b0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.726+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1405f060 con 0x7f5b141015b0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.726+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f5b04009440 con 0x7f5b141015b0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.726+0000 7f5b09ffb700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5b00036660 0x7f5b00038b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.727+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5b04049610 con 0x7f5b141015b0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.727+0000 7f5b18b07700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5b00036660 0x7f5b00038b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.727+0000 7f5b18b07700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5b00036660 0x7f5b00038b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009990 tx=0x7f5b10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:34.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:34 smithi133 systemd[1]: Starting Ceph mon.c for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:04:34.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:34.732+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f5b04011550 con 0x7f5b141015b0 2023-06-22T18:04:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.014+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5b14105750 con 0x7f5b141015b0 2023-06-22T18:04:35.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.015+0000 7f5b09ffb700 1 -- 172.21.15.133:0/1462169073 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+731 (secure 0 0 0) 0x7f5b04011370 con 0x7f5b141015b0 2023-06-22T18:04:35.020 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:04:35.020 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","modified":"2023-06-22T18:02:46.893669Z","created":"2023-06-22T18:02:46.893669Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-22T18:04:35.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5b00036660 msgr2=0x7f5b00038b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:35.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5b00036660 0x7f5b00038b10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009990 tx=0x7f5b10006e40 comp rx=0 tx=0).stop 2023-06-22T18:04:35.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 msgr2=0x7f5b141116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:35.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b141116d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400c010 tx=0x7f5b04009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:35.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 shutdown_connections 2023-06-22T18:04:35.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5b00036660 0x7f5b00038b10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:35.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 --2- 172.21.15.133:0/1462169073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b141015b0 0x7f5b141116d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:35.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 >> 172.21.15.133:0/1462169073 conn(0x7f5b140713d0 msgr2=0x7f5b140f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:35.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.017+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 shutdown_connections 2023-06-22T18:04:35.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:35.018+0000 7f5b1b56c700 1 -- 172.21.15.133:0/1462169073 wait complete. 2023-06-22T18:04:35.023 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 1 2023-06-22T18:04:35.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:34 smithi039 ceph-mon[109976]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.668 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: set uid:gid to 167:167 (ceph:ceph) 2023-06-22T18:04:35.668 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable), process ceph-mon, pid 7 2023-06-22T18:04:35.668 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pidfile_write: ignore empty --pid-file 2023-06-22T18:04:35.669 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: load: jerasure load: lrc 2023-06-22T18:04:35.669 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: RocksDB version: 6.15.5 2023-06-22T18:04:35.669 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-06-22T18:04:35.669 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Compile date Jun 21 2023 2023-06-22T18:04:35.669 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: DB SUMMARY 2023-06-22T18:04:35.669 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: DB Session ID: NSVCATTCAZ3HTW1NPBHR 2023-06-22T18:04:35.670 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: CURRENT file: CURRENT 2023-06-22T18:04:35.670 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: IDENTITY file: IDENTITY 2023-06-22T18:04:35.670 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-06-22T18:04:35.670 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-06-22T18:04:35.670 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-06-22T18:04:35.670 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.error_if_exists: 0 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.create_if_missing: 0 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.paranoid_checks: 1 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.env: 0x563bc2a57a80 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.fs: Posix File System 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.info_log: 0x563bc3a8dda0 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 bash[116783]: e18ee63705e73b312e051add4b6f8febe17859af775771547c4f46b783b2ffec 2023-06-22T18:04:35.671 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_file_opening_threads: 16 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.statistics: (nil) 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.use_fsync: 0 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_log_file_size: 0 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.keep_log_file_num: 1000 2023-06-22T18:04:35.672 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.recycle_log_file_num: 0 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.allow_fallocate: 1 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.allow_mmap_reads: 0 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.allow_mmap_writes: 0 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.use_direct_reads: 0 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.create_missing_column_families: 0 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.db_log_dir: 2023-06-22T18:04:35.673 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.advise_random_on_open: 1 2023-06-22T18:04:35.674 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.db_write_buffer_size: 0 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.write_buffer_manager: 0x563bc3b54930 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.rate_limiter: (nil) 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-22T18:04:35.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.wal_recovery_mode: 2 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.enable_thread_tracking: 0 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.enable_pipelined_write: 0 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.unordered_write: 0 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-22T18:04:35.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.row_cache: None 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.wal_filter: None 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.allow_ingest_behind: 0 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.preserve_deletes: 0 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.two_write_queues: 0 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.manual_wal_flush: 0 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.atomic_flush: 0 2023-06-22T18:04:35.677 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.log_readahead_size: 0 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.best_efforts_recovery: 0 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-22T18:04:35.678 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.allow_data_in_errors: 0 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.db_host_id: __hostname__ 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_background_jobs: 2 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_background_compactions: -1 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_subcompactions: 1 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-22T18:04:35.679 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_total_wal_size: 0 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_open_files: -1 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bytes_per_sync: 0 2023-06-22T18:04:35.680 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_readahead_size: 0 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_background_flushes: -1 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Compression algorithms supported: 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kZSTD supported: 0 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kXpressCompression supported: 0 2023-06-22T18:04:35.681 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kLZ4HCCompression supported: 1 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kZlibCompression supported: 1 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kSnappyCompression supported: 1 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kLZ4Compression supported: 1 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: kBZip2Compression supported: 0 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-06-22T18:04:35.682 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.merge_operator: 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_filter: None 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_filter_factory: None 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.sst_partitioner_factory: None 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563bc3a58d30) 2023-06-22T18:04:35.683 INFO:journalctl@ceph.mon.c.smithi133.stdout: cache_index_and_filter_blocks: 1 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: pin_top_level_index_and_filter: 1 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: index_type: 0 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: data_block_index_type: 0 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: index_shortening: 1 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-22T18:04:35.684 INFO:journalctl@ceph.mon.c.smithi133.stdout: hash_index_allow_collision: 1 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: checksum: 1 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: no_block_cache: 0 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache: 0x563bc3b171f0 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache_name: BinnedLRUCache 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache_options: 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: capacity : 536870912 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: num_shard_bits : 4 2023-06-22T18:04:35.685 INFO:journalctl@ceph.mon.c.smithi133.stdout: strict_capacity_limit : 0 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: high_pri_pool_ratio: 0.000 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache_compressed: (nil) 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: persistent_cache: (nil) 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_size: 4096 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_size_deviation: 10 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_restart_interval: 16 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: index_block_restart_interval: 1 2023-06-22T18:04:35.686 INFO:journalctl@ceph.mon.c.smithi133.stdout: metadata_block_size: 4096 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: partition_filters: 0 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: use_delta_encoding: 1 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: whole_key_filtering: 1 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: verify_compression: 0 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: read_amp_bytes_per_bit: 0 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: format_version: 4 2023-06-22T18:04:35.687 INFO:journalctl@ceph.mon.c.smithi133.stdout: enable_index_compression: 1 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_align: 0 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.write_buffer_size: 33554432 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_write_buffer_number: 2 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression: NoCompression 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression: Disabled 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.prefix_extractor: nullptr 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-22T18:04:35.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.num_levels: 7 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-22T18:04:35.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.level: 32767 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.strategy: 0 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-22T18:04:35.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compression_opts.enabled: false 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.target_file_size_base: 67108864 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-22T18:04:35.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-22T18:04:35.692 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.arena_block_size: 4194304 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-06-22T18:04:35.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.disable_auto_compactions: 0 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-22T18:04:35.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.table_properties_collectors: 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.inplace_update_support: 0 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-22T18:04:35.695 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.bloom_locality: 0 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.max_successive_merges: 0 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.paranoid_file_checks: 0 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.force_consistency_checks: 1 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.report_bg_io_stats: 0 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.ttl: 2592000 2023-06-22T18:04:35.696 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.enable_blob_files: false 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.min_blob_size: 0 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.blob_file_size: 268435456 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-22T18:04:35.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687457075248961, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687457075249217, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1687457075, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "8754d522-506d-49d5-9ccd-14f5a228504c", "db_session_id": "NSVCATTCAZ3HTW1NPBHR"}} 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687457075249528, "job": 1, "event": "recovery_finished"} 2023-06-22T18:04:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x563bc3aab180 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: DB pointer 0x563bc3ac6000 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** DB Stats ** 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:04:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:35.701 INFO:journalctl@ceph.mon.c.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-22T18:04:35.702 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:35.703 INFO:journalctl@ceph.mon.c.smithi133.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:35.704 INFO:journalctl@ceph.mon.c.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.2 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:35.705 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:35.706 INFO:journalctl@ceph.mon.c.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-22T18:04:35.706 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: using public_addr v2:172.21.15.133:0/0 -> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: starting mon.c rank -1 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(???) e0 preinit fsid d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).mds e1 new map 2023-06-22T18:04:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).mds e1 print_map 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout: e1 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout: legacy client fscid: -1 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout: No filesystems configured 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-06-22T18:04:35.708 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:04:35.709 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:35.710 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi101:/etc/ceph/ceph.conf 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:35.711 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Deploying cephadm binary to smithi133 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Added host smithi133 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.713 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi133:/etc/ceph/ceph.conf 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1872430045' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:35.714 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1872430045' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:35.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:35.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c;count:3 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: Deploying daemon mon.c on smithi133 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 ceph-mon[116867]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-06-22T18:04:35.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:35 smithi133 systemd[1]: Started Ceph mon.c for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:04:36.945 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-22T18:04:36.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-22T18:04:37.533 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:04:40.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 -- 172.21.15.133:0/1794760366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f01027c0 msgr2=0x7f92f0102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:40.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 --2- 172.21.15.133:0/1794760366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f01027c0 0x7f92f0102b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009ce0 tx=0x7f92ec00b040 comp rx=0 tx=0).stop 2023-06-22T18:04:40.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 -- 172.21.15.133:0/1794760366 shutdown_connections 2023-06-22T18:04:40.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 --2- 172.21.15.133:0/1794760366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f01027c0 0x7f92f0102b90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:40.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 -- 172.21.15.133:0/1794760366 >> 172.21.15.133:0/1794760366 conn(0x7f92f00f8340 msgr2=0x7f92f00fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:40.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 -- 172.21.15.133:0/1794760366 shutdown_connections 2023-06-22T18:04:40.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.617+0000 7f92f872c700 1 -- 172.21.15.133:0/1794760366 wait complete. 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f872c700 1 Processor -- start 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f872c700 1 -- start start 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f872c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f872c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f0113130 0x7f92f010cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f872c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f01136b0 con 0x7f92f0113130 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f872c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f0113820 con 0x7f92f01027c0 2023-06-22T18:04:40.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47420/0 (socket says 172.21.15.133:47420) 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 -- 172.21.15.133:0/2845562955 learned_addr learned my addr 172.21.15.133:0/2845562955 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 -- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 msgr2=0x7f92f0112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 -- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 msgr2=0x7f92f0112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.618+0000 7f92f64c8700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92f5cc7700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f0113130 0x7f92f010cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:40.625 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92f5cc7700 1 -- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 msgr2=0x7f92f0112bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92f5cc7700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92f5cc7700 1 -- 172.21.15.133:0/2845562955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec005040 con 0x7f92f0113130 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92f5cc7700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f0113130 0x7f92f010cad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92e000b7c0 tx=0x7f92e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f92e00117e0 con 0x7f92f0113130 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e0011e10 con 0x7f92f0113130 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.619+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f010d070 con 0x7f92f0113130 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.620+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f92e0010e80 con 0x7f92f0113130 2023-06-22T18:04:40.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.620+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f010d630 con 0x7f92f0113130 2023-06-22T18:04:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.620+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f92e001a400 con 0x7f92f0113130 2023-06-22T18:04:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.621+0000 7f92e6ffd700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f92dc0367f0 0x7f92dc038ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:40.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.621+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f010d910 con 0x7f92f0113130 2023-06-22T18:04:40.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.621+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f92e00486e0 con 0x7f92f0113130 2023-06-22T18:04:40.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.621+0000 7f92f64c8700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f92dc0367f0 0x7f92dc038ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:40.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.622+0000 7f92f64c8700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f92dc0367f0 0x7f92dc038ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f92ec004730 tx=0x7f92ec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:40.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.627+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f92e0015030 con 0x7f92f0113130 2023-06-22T18:04:40.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:04:40.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: mon.a calling monitor election 2023-06-22T18:04:40.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:40.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: mon.c calling monitor election 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: fsmap 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: mgrmap e13: x(active, since 46s) 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: overall HEALTH_OK 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:40.697 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:40.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:40.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:40.698 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:04:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: mon.a calling monitor election 2023-06-22T18:04:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: mon.c calling monitor election 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: fsmap 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:40.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: mgrmap e13: x(active, since 46s) 2023-06-22T18:04:40.737 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: overall HEALTH_OK 2023-06-22T18:04:40.737 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:40.737 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:40.737 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:40.737 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:40.737 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.886+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f92f005f060 con 0x7f92f0113130 2023-06-22T18:04:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.887+0000 7f92e6ffd700 1 -- 172.21.15.133:0/2845562955 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1001 (secure 0 0 0) 0x7f92e0046020 con 0x7f92f0113130 2023-06-22T18:04:40.892 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:04:40.892 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","modified":"2023-06-22T18:04:35.271414Z","created":"2023-06-22T18:02:46.893669Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-06-22T18:04:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f92dc0367f0 msgr2=0x7f92dc038ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f92dc0367f0 0x7f92dc038ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f92ec004730 tx=0x7f92ec00b040 comp rx=0 tx=0).stop 2023-06-22T18:04:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f0113130 msgr2=0x7f92f010cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f0113130 0x7f92f010cad0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92e000b7c0 tx=0x7f92e00091a0 comp rx=0 tx=0).stop 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 shutdown_connections 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f92dc0367f0 0x7f92dc038ca0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f01027c0 0x7f92f0112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 --2- 172.21.15.133:0/2845562955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f0113130 0x7f92f010cad0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 >> 172.21.15.133:0/2845562955 conn(0x7f92f00f8340 msgr2=0x7f92f00f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 shutdown_connections 2023-06-22T18:04:40.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:40.889+0000 7f92f872c700 1 -- 172.21.15.133:0/2845562955 wait complete. 2023-06-22T18:04:40.895 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2023-06-22T18:04:41.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:41 smithi039 ceph-mon[109976]: Deploying daemon mon.b on smithi101 2023-06-22T18:04:41.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:41 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/2845562955' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-22T18:04:41.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:41 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:41.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:41 smithi133 ceph-mon[116867]: Deploying daemon mon.b on smithi101 2023-06-22T18:04:41.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:41 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/2845562955' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-22T18:04:41.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:41 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:42.570 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-22T18:04:42.571 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-22T18:04:42.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:42 smithi039 ceph-mon[109976]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:42.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:42 smithi133 ceph-mon[116867]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:43.160 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:04:43.788 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 systemd[1]: Starting Ceph mon.b for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:04:44.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: set uid:gid to 167:167 (ceph:ceph) 2023-06-22T18:04:44.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: ceph version 17.2.6-719-g42ddc46e (42ddc46e6e6582d8e29a5a982a81886fe028be2c) quincy (stable), process ceph-mon, pid 7 2023-06-22T18:04:44.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pidfile_write: ignore empty --pid-file 2023-06-22T18:04:44.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: load: jerasure load: lrc 2023-06-22T18:04:44.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: RocksDB version: 6.15.5 2023-06-22T18:04:44.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-06-22T18:04:44.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Compile date Jun 21 2023 2023-06-22T18:04:44.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: DB SUMMARY 2023-06-22T18:04:44.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: DB Session ID: 6LS2R0GIYJNYKMEJYSVD 2023-06-22T18:04:44.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: CURRENT file: CURRENT 2023-06-22T18:04:44.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: IDENTITY file: IDENTITY 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.error_if_exists: 0 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.create_if_missing: 0 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.paranoid_checks: 1 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-22T18:04:44.359 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.env: 0x56140acfea80 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.fs: Posix File System 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.info_log: 0x56140c2edda0 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_file_opening_threads: 16 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.statistics: (nil) 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.use_fsync: 0 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_log_file_size: 0 2023-06-22T18:04:44.360 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.keep_log_file_num: 1000 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.recycle_log_file_num: 0 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.allow_fallocate: 1 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.allow_mmap_reads: 0 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.allow_mmap_writes: 0 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.use_direct_reads: 0 2023-06-22T18:04:44.361 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.create_missing_column_families: 0 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.db_log_dir: 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-22T18:04:44.362 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.advise_random_on_open: 1 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.db_write_buffer_size: 0 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.write_buffer_manager: 0x56140c3b48d0 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-22T18:04:44.363 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-22T18:04:44.364 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.rate_limiter: (nil) 2023-06-22T18:04:44.364 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-22T18:04:44.364 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.wal_recovery_mode: 2 2023-06-22T18:04:44.364 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.enable_thread_tracking: 0 2023-06-22T18:04:44.365 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.enable_pipelined_write: 0 2023-06-22T18:04:44.365 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.unordered_write: 0 2023-06-22T18:04:44.365 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-22T18:04:44.365 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-22T18:04:44.366 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-22T18:04:44.366 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-22T18:04:44.366 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.row_cache: None 2023-06-22T18:04:44.366 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.wal_filter: None 2023-06-22T18:04:44.367 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-22T18:04:44.367 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.allow_ingest_behind: 0 2023-06-22T18:04:44.367 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.preserve_deletes: 0 2023-06-22T18:04:44.368 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 bash[116203]: d72d39d5744d1be633856b1384bfb22c112eadfb81234a9aae9ab1873e15dbdd 2023-06-22T18:04:44.368 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.two_write_queues: 0 2023-06-22T18:04:44.368 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.manual_wal_flush: 0 2023-06-22T18:04:44.369 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.atomic_flush: 0 2023-06-22T18:04:44.369 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-22T18:04:44.369 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-22T18:04:44.369 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-22T18:04:44.370 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.log_readahead_size: 0 2023-06-22T18:04:44.370 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-22T18:04:44.370 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.best_efforts_recovery: 0 2023-06-22T18:04:44.371 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-22T18:04:44.371 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-22T18:04:44.371 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.allow_data_in_errors: 0 2023-06-22T18:04:44.371 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.db_host_id: __hostname__ 2023-06-22T18:04:44.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_background_jobs: 2 2023-06-22T18:04:44.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_background_compactions: -1 2023-06-22T18:04:44.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_subcompactions: 1 2023-06-22T18:04:44.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-22T18:04:44.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-22T18:04:44.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-22T18:04:44.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_total_wal_size: 0 2023-06-22T18:04:44.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-22T18:04:44.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-22T18:04:44.374 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-22T18:04:44.374 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-22T18:04:44.374 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_open_files: -1 2023-06-22T18:04:44.375 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bytes_per_sync: 0 2023-06-22T18:04:44.375 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-22T18:04:44.375 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-22T18:04:44.375 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_readahead_size: 0 2023-06-22T18:04:44.375 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_background_flushes: -1 2023-06-22T18:04:44.375 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Compression algorithms supported: 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kZSTD supported: 0 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kXpressCompression supported: 0 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kLZ4HCCompression supported: 1 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kZlibCompression supported: 1 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kSnappyCompression supported: 1 2023-06-22T18:04:44.376 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kLZ4Compression supported: 1 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: kBZip2Compression supported: 0 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.merge_operator: 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_filter: None 2023-06-22T18:04:44.377 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_filter_factory: None 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.sst_partitioner_factory: None 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56140c2b8d30) 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout: cache_index_and_filter_blocks: 1 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-22T18:04:44.378 INFO:journalctl@ceph.mon.b.smithi101.stdout: pin_top_level_index_and_filter: 1 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: index_type: 0 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: data_block_index_type: 0 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: index_shortening: 1 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: hash_index_allow_collision: 1 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: checksum: 1 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: no_block_cache: 0 2023-06-22T18:04:44.379 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache: 0x56140c3771f0 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache_name: BinnedLRUCache 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache_options: 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: capacity : 536870912 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: num_shard_bits : 4 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: strict_capacity_limit : 0 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: high_pri_pool_ratio: 0.000 2023-06-22T18:04:44.380 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache_compressed: (nil) 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: persistent_cache: (nil) 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_size: 4096 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_size_deviation: 10 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_restart_interval: 16 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: index_block_restart_interval: 1 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: metadata_block_size: 4096 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: partition_filters: 0 2023-06-22T18:04:44.381 INFO:journalctl@ceph.mon.b.smithi101.stdout: use_delta_encoding: 1 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: whole_key_filtering: 1 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: verify_compression: 0 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: read_amp_bytes_per_bit: 0 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: format_version: 4 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: enable_index_compression: 1 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_align: 0 2023-06-22T18:04:44.382 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.write_buffer_size: 33554432 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_write_buffer_number: 2 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression: NoCompression 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression: Disabled 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.prefix_extractor: nullptr 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.num_levels: 7 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-22T18:04:44.383 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-22T18:04:44.384 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.level: 32767 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.strategy: 0 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-22T18:04:44.385 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compression_opts.enabled: false 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.target_file_size_base: 67108864 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-22T18:04:44.386 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-22T18:04:44.387 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.arena_block_size: 4194304 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.disable_auto_compactions: 0 2023-06-22T18:04:44.388 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-22T18:04:44.389 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.table_properties_collectors: 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.inplace_update_support: 0 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-22T18:04:44.390 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.bloom_locality: 0 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.max_successive_merges: 0 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.paranoid_file_checks: 0 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.force_consistency_checks: 1 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.report_bg_io_stats: 0 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.ttl: 2592000 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-22T18:04:44.391 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.enable_blob_files: false 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.min_blob_size: 0 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.blob_file_size: 268435456 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2023-06-22T18:04:44.392 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687457083959428, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687457083959703, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1670, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1687457083, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "067e9e3a-60f2-4c43-b7ea-2257b9aa27d1", "db_session_id": "6LS2R0GIYJNYKMEJYSVD"}} 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687457083960008, "job": 1, "event": "recovery_finished"} 2023-06-22T18:04:44.393 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x56140c30b180 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: DB pointer 0x56140c326000 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** DB Stats ** 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-22T18:04:44.394 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:44.395 INFO:journalctl@ceph.mon.b.smithi101.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:44.396 INFO:journalctl@ceph.mon.b.smithi101.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:44.397 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:44.398 INFO:journalctl@ceph.mon.b.smithi101.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** Compaction Stats [default] ** 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.1 0.00 0.00 1 0.000 0 0 2023-06-22T18:04:44.399 INFO:journalctl@ceph.mon.b.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:44.400 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: using public_addr v2:172.21.15.101:0/0 -> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: starting mon.b rank -1 at public addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] at bind addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(???) e0 preinit fsid d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:44.401 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).mds e1 new map 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).mds e1 print_map 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout: e1 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout: legacy client fscid: -1 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout: No filesystems configured 2023-06-22T18:04:44.402 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:04:44.403 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:44.404 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi101:/etc/ceph/ceph.conf 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:44.405 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Deploying cephadm binary to smithi133 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Added host smithi133 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.406 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.407 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi133:/etc/ceph/ceph.conf 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1872430045' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-22T18:04:44.408 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1872430045' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-22T18:04:44.409 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:44.410 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi101:172.21.15.101=b;smithi133:172.21.15.133=c;count:3 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Deploying daemon mon.c on smithi133 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.a calling monitor election 2023-06-22T18:04:44.411 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.c calling monitor election 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.412 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: monmap e2: 2 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: fsmap 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mgrmap e13: x(active, since 46s) 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: overall HEALTH_OK 2023-06-22T18:04:44.413 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: Deploying daemon mon.b on smithi101 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/2845562955' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:44.414 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:44.415 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-06-22T18:04:44.415 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:43 smithi101 ceph-mon[116267]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-06-22T18:04:44.415 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:44 smithi101 systemd[1]: Started Ceph mon.b for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:04:49.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3980890660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff370003580 msgr2=0x7ff370005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:49.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3980890660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff370003580 0x7ff370005a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff3840fc6b0 tx=0x7ff380002040 comp rx=0 tx=0).stop 2023-06-22T18:04:49.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3980890660 shutdown_connections 2023-06-22T18:04:49.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3980890660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff370003580 0x7ff370005a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3980890660 >> 172.21.15.133:0/3980890660 conn(0x7ff3840f8340 msgr2=0x7ff3840fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3980890660 shutdown_connections 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.066+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3980890660 wait complete. 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 Processor -- start 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 -- start start 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 0x7ff38406fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:49.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff384118c20 0x7ff38411b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38411b7f0 con 0x7ff384118c20 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38411b960 con 0x7ff370003580 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38bf6b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38411bb20 con 0x7ff384070270 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff389506700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff389506700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47448/0 (socket says 172.21.15.133:47448) 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff389506700 1 -- 172.21.15.133:0/3137129872 learned_addr learned my addr 172.21.15.133:0/3137129872 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:04:49.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.067+0000 7ff38a508700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff384118c20 0x7ff38411b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389506700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 msgr2=0x7ff3840706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389506700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 msgr2=0x7ff3840706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389506700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389d07700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 0x7ff38406fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389506700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389d07700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 msgr2=0x7ff38406fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-06-22T18:04:49.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389d07700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 msgr2=0x7ff38406fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389d07700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 0x7ff38406fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff389d07700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 0x7ff38406fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38a508700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 msgr2=0x7ff3840706e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38a508700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38a508700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 msgr2=0x7ff38406fd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38a508700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 0x7ff38406fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38a508700 1 -- 172.21.15.133:0/3137129872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380005040 con 0x7ff384118c20 2023-06-22T18:04:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38a508700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff384118c20 0x7ff38411b020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00bad0 tx=0x7ff37c009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00c8c0 con 0x7ff384118c20 2023-06-22T18:04:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38411be00 con 0x7ff384118c20 2023-06-22T18:04:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.068+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff37c0098a0 con 0x7ff384118c20 2023-06-22T18:04:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.069+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0133c0 con 0x7ff384118c20 2023-06-22T18:04:49.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.069+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38411c210 con 0x7ff384118c20 2023-06-22T18:04:49.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.069+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7ff37c0137f0 con 0x7ff384118c20 2023-06-22T18:04:49.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.070+0000 7ff37a7fc700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff368036b30 0x7ff368038fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:49.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.070+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38405f060 con 0x7ff384118c20 2023-06-22T18:04:49.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.070+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff37c0484a0 con 0x7ff384118c20 2023-06-22T18:04:49.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.070+0000 7ff389d07700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff368036b30 0x7ff368038fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:49.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.071+0000 7ff389d07700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff368036b30 0x7ff368038fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff3800047e0 tx=0x7ff38000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:49.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.075+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff37c001900 con 0x7ff384118c20 2023-06-22T18:04:49.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.346+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff3841043c0 con 0x7ff384118c20 2023-06-22T18:04:49.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.347+0000 7ff37a7fc700 1 -- 172.21.15.133:0/3137129872 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1269 (secure 0 0 0) 0x7ff37c020120 con 0x7ff384118c20 2023-06-22T18:04:49.351 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-06-22T18:04:49.351 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","modified":"2023-06-22T18:04:43.980005Z","created":"2023-06-22T18:02:46.893669Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:3300","nonce":0},{"type":"v1","addr":"172.21.15.101:6789","nonce":0}]},"addr":"172.21.15.101:6789/0","public_addr":"172.21.15.101:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-06-22T18:04:49.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff368036b30 msgr2=0x7ff368038fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:49.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff368036b30 0x7ff368038fe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff3800047e0 tx=0x7ff38000b040 comp rx=0 tx=0).stop 2023-06-22T18:04:49.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff384118c20 msgr2=0x7ff38411b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:49.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff384118c20 0x7ff38411b020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00bad0 tx=0x7ff37c009760 comp rx=0 tx=0).stop 2023-06-22T18:04:49.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 shutdown_connections 2023-06-22T18:04:49.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff368036b30 0x7ff368038fe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff384118c20 0x7ff38411b020 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff370003580 0x7ff38406fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.349+0000 7ff38bf6b700 1 --2- 172.21.15.133:0/3137129872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff384070270 0x7ff3840706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.350+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 >> 172.21.15.133:0/3137129872 conn(0x7ff3840f8340 msgr2=0x7ff384100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.350+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 shutdown_connections 2023-06-22T18:04:49.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:49.350+0000 7ff38bf6b700 1 -- 172.21.15.133:0/3137129872 wait complete. 2023-06-22T18:04:49.355 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 3 2023-06-22T18:04:49.385 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:49.385 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:04:49.385 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: mon.a calling monitor election 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: mon.c calling monitor election 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.386 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: fsmap 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:49.387 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: mgrmap e13: x(active, since 55s) 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: mon.b (rank 2) addr [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] is down (out of quorum) 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:49.388 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:04:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: mon.a calling monitor election 2023-06-22T18:04:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: mon.c calling monitor election 2023-06-22T18:04:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:49.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-22T18:04:49.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:49.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: fsmap 2023-06-22T18:04:49.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:49.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: mgrmap e13: x(active, since 55s) 2023-06-22T18:04:49.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-06-22T18:04:49.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-06-22T18:04:49.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-06-22T18:04:49.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: mon.b (rank 2) addr [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] is down (out of quorum) 2023-06-22T18:04:49.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:49.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:49.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:49.994 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-06-22T18:04:49.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph config generate-minimal-conf 2023-06-22T18:04:50.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:50.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:50.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:50 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/3137129872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-22T18:04:50.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:50 smithi133 ceph-mon[116867]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:50.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:50.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:50.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:50.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:50 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/3137129872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-22T18:04:50.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:50 smithi039 ceph-mon[109976]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:50.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:50.773 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:04:51.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: mon.a calling monitor election 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: mon.c calling monitor election 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-22T18:04:51.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:51.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: fsmap 2023-06-22T18:04:51.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:51.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: mgrmap e13: x(active, since 55s) 2023-06-22T18:04:51.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-06-22T18:04:51.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-06-22T18:04:51.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-06-22T18:04:51.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: mon.b (rank 2) addr [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] is down (out of quorum) 2023-06-22T18:04:51.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:51.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/3137129872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-22T18:04:51.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:51.358 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: mon.b calling monitor election 2023-06-22T18:04:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: Updating smithi039:/etc/ceph/ceph.conf 2023-06-22T18:04:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: Updating smithi101:/etc/ceph/ceph.conf 2023-06-22T18:04:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: Updating smithi133:/etc/ceph/ceph.conf 2023-06-22T18:04:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: mon.b calling monitor election 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: mon.c calling monitor election 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: mon.a calling monitor election 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: fsmap 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:51.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: mgrmap e13: x(active, since 57s) 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: Cluster is now healthy 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: overall HEALTH_OK 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:51.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: mon.b calling monitor election 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: Updating smithi039:/etc/ceph/ceph.conf 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: Updating smithi101:/etc/ceph/ceph.conf 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: Updating smithi133:/etc/ceph/ceph.conf 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: mon.b calling monitor election 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: mon.c calling monitor election 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: mon.a calling monitor election 2023-06-22T18:04:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: fsmap 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: mgrmap e13: x(active, since 57s) 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: Cluster is now healthy 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: overall HEALTH_OK 2023-06-22T18:04:52.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.238 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: mon.b calling monitor election 2023-06-22T18:04:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: Updating smithi039:/etc/ceph/ceph.conf 2023-06-22T18:04:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: Updating smithi101:/etc/ceph/ceph.conf 2023-06-22T18:04:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: Updating smithi133:/etc/ceph/ceph.conf 2023-06-22T18:04:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: mon.b calling monitor election 2023-06-22T18:04:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: mon.c calling monitor election 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: mon.a calling monitor election 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: monmap e3: 3 mons at {a=[v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: fsmap 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: osdmap e4: 0 total, 0 up, 0 in 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: mgrmap e13: x(active, since 57s) 2023-06-22T18:04:52.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: Cluster is now healthy 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: overall HEALTH_OK 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:52.445 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 -- 172.21.15.39:0/1280056281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c0fd680 msgr2=0x7ff40c0fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 --2- 172.21.15.39:0/1280056281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c0fd680 0x7ff40c0fda50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff4080048c0 tx=0x7ff408009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 -- 172.21.15.39:0/1280056281 shutdown_connections 2023-06-22T18:04:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 --2- 172.21.15.39:0/1280056281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c0fd680 0x7ff40c0fda50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 -- 172.21.15.39:0/1280056281 >> 172.21.15.39:0/1280056281 conn(0x7ff40c0f7100 msgr2=0x7ff40c0f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 -- 172.21.15.39:0/1280056281 shutdown_connections 2023-06-22T18:04:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.441+0000 7ff4132df700 1 -- 172.21.15.39:0/1280056281 wait complete. 2023-06-22T18:04:52.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 Processor -- start 2023-06-22T18:04:52.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 -- start start 2023-06-22T18:04:52.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff40c0fd680 0x7ff40c112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:52.448 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff40c113260 0x7ff40c10cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:52.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 0x7ff40c10d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:52.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c1138c0 con 0x7ff40c10d100 2023-06-22T18:04:52.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c119b50 con 0x7ff40c113260 2023-06-22T18:04:52.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.442+0000 7ff4132df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c119c90 con 0x7ff40c0fd680 2023-06-22T18:04:52.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 0x7ff40c10d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:52.449 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41107b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff40c0fd680 0x7ff40c112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 0x7ff40c10d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42412/0 (socket says 172.21.15.39:42412) 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 -- 172.21.15.39:0/1787421710 learned_addr learned my addr 172.21.15.39:0/1787421710 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41087a700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff40c113260 0x7ff40c10cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 -- 172.21.15.39:0/1787421710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff40c0fd680 msgr2=0x7ff40c112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff40c0fd680 0x7ff40c112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 -- 172.21.15.39:0/1787421710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff40c113260 msgr2=0x7ff40c10cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:52.450 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff40c113260 0x7ff40c10cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 -- 172.21.15.39:0/1787421710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408005040 con 0x7ff40c10d100 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41107b700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff40c0fd680 0x7ff40c112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.443+0000 7ff41187c700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 0x7ff40c10d5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff40400bc30 tx=0x7ff404009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.444+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40400c840 con 0x7ff40c10d100 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.444+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff404009960 con 0x7ff40c10d100 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.444+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c119f40 con 0x7ff40c10d100 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.444+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4040073e0 con 0x7ff40c10d100 2023-06-22T18:04:52.451 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.444+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c11a3a0 con 0x7ff40c10d100 2023-06-22T18:04:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.445+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7ff40400c9a0 con 0x7ff40c10d100 2023-06-22T18:04:52.452 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.445+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c05f060 con 0x7ff40c10d100 2023-06-22T18:04:52.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.445+0000 7ff401ffb700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff3f8036b20 0x7ff3f8038fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:52.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.445+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff404048470 con 0x7ff40c10d100 2023-06-22T18:04:52.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.446+0000 7ff41107b700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff3f8036b20 0x7ff3f8038fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:52.453 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.446+0000 7ff41107b700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff3f8036b20 0x7ff3f8038fd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff40800bb80 tx=0x7ff408009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:52.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.450+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff404001b00 con 0x7ff40c10d100 2023-06-22T18:04:52.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.666+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff40c103e50 con 0x7ff40c10d100 2023-06-22T18:04:52.671 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.667+0000 7ff401ffb700 1 -- 172.21.15.39:0/1787421710 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7ff40401b310 con 0x7ff40c10d100 2023-06-22T18:04:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:52.672 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2023-06-22T18:04:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = d572fea0-1126-11ee-9b2b-001a4aab830c 2023-06-22T18:04:52.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2023-06-22T18:04:52.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff3f8036b20 msgr2=0x7ff3f8038fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff3f8036b20 0x7ff3f8038fd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff40800bb80 tx=0x7ff408009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 msgr2=0x7ff40c10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 0x7ff40c10d5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff40400bc30 tx=0x7ff404009820 comp rx=0 tx=0).stop 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 shutdown_connections 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff3f8036b20 0x7ff3f8038fd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff40c10d100 0x7ff40c10d5a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff40c0fd680 0x7ff40c112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 --2- 172.21.15.39:0/1787421710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff40c113260 0x7ff40c10cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:52.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.669+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 >> 172.21.15.39:0/1787421710 conn(0x7ff40c0f7100 msgr2=0x7ff40c100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:52.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.670+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 shutdown_connections 2023-06-22T18:04:52.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:04:52.670+0000 7ff4132df700 1 -- 172.21.15.39:0/1787421710 wait complete. 2023-06-22T18:04:53.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: Updating smithi039:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: Reconfiguring mon.a (unknown last config time)... 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: Reconfiguring daemon mon.a on smithi039 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:53.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:52 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1787421710' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:53.146 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-06-22T18:04:53.147 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:53.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2023-06-22T18:04:53.216 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:53.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: Updating smithi039:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: Reconfiguring mon.a (unknown last config time)... 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: Reconfiguring daemon mon.a on smithi039 2023-06-22T18:04:53.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:53.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:52 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1787421710' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:53.318 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:04:53.319 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.conf 2023-06-22T18:04:53.347 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: Updating smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.347 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: Updating smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: Updating smithi039:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: Reconfiguring mon.a (unknown last config time)... 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:53.348 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: Reconfiguring daemon mon.a on smithi039 2023-06-22T18:04:53.349 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:04:53.349 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:52 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1787421710' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:53.352 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:04:53.352 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:53.419 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:04:53.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2023-06-22T18:04:53.451 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:04:53.452 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-22T18:04:53.518 INFO:tasks.cephadm:Adding mgr.x on smithi039 2023-06-22T18:04:53.519 INFO:tasks.cephadm:Adding mgr.y on smithi101 2023-06-22T18:04:53.519 INFO:tasks.cephadm:Adding mgr.z on smithi133 2023-06-22T18:04:53.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch apply mgr '3;smithi039=x;smithi101=y;smithi133=z' 2023-06-22T18:04:54.136 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:04:54.706 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: Reconfiguring mon.b (monmap changed)... 2023-06-22T18:04:54.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:54.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:54.708 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:54.708 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: Reconfiguring daemon mon.b on smithi101 2023-06-22T18:04:54.708 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:54.708 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: Reconfiguring mon.b (monmap changed)... 2023-06-22T18:04:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: Reconfiguring daemon mon.b on smithi101 2023-06-22T18:04:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: Reconfiguring mon.b (monmap changed)... 2023-06-22T18:04:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: Reconfiguring daemon mon.b on smithi101 2023-06-22T18:04:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:55.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- 172.21.15.133:0/3439679704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33200fc990 msgr2=0x7f33200fcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:55.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 --2- 172.21.15.133:0/3439679704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33200fc990 0x7f33200fcda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f331c0048c0 tx=0x7f331c009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:55.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- 172.21.15.133:0/3439679704 shutdown_connections 2023-06-22T18:04:55.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 --2- 172.21.15.133:0/3439679704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33200fc990 0x7f33200fcda0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- 172.21.15.133:0/3439679704 >> 172.21.15.133:0/3439679704 conn(0x7f33200f8050 msgr2=0x7f33200fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- 172.21.15.133:0/3439679704 shutdown_connections 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- 172.21.15.133:0/3439679704 wait complete. 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 Processor -- start 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- start start 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33200fc990 0x7f332010cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f332010d400 0x7f332010e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 0x7f332010eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:55.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320115410 con 0x7f332010d400 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320115550 con 0x7f33200fc990 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.655+0000 7f3328096700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320115690 con 0x7f332010e700 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 0x7f332010eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 0x7f332010eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47460/0 (socket says 172.21.15.133:47460) 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 -- 172.21.15.133:0/473842379 learned_addr learned my addr 172.21.15.133:0/473842379 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3325631700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f332010d400 0x7f332010e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:55.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3325e32700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33200fc990 0x7f332010cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 -- 172.21.15.133:0/473842379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33200fc990 msgr2=0x7f332010cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33200fc990 0x7f332010cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 -- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f332010d400 msgr2=0x7f332010e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f332010d400 0x7f332010e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 -- 172.21.15.133:0/473842379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c005040 con 0x7f332010e700 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3326633700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 0x7f332010eba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f331800bc30 tx=0x7f3318009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800c840 con 0x7f332010e700 2023-06-22T18:04:55.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f331800ce70 con 0x7f332010e700 2023-06-22T18:04:55.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33201158c0 con 0x7f332010e700 2023-06-22T18:04:55.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33180073e0 con 0x7f332010e700 2023-06-22T18:04:55.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.656+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3320115d80 con 0x7f332010e700 2023-06-22T18:04:55.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.657+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f3318007810 con 0x7f332010e700 2023-06-22T18:04:55.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.657+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332004b970 con 0x7f332010e700 2023-06-22T18:04:55.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.657+0000 7f33167fc700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f330c036ad0 0x7f330c038f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:04:55.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.657+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f33180486f0 con 0x7f332010e700 2023-06-22T18:04:55.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.658+0000 7f3325e32700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f330c036ad0 0x7f330c038f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:04:55.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.658+0000 7f3325e32700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f330c036ad0 0x7f330c038f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f331c00b3b0 tx=0x7f331c009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:04:55.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.662+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f3318001310 con 0x7f332010e700 2023-06-22T18:04:55.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.866+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi101=y;smithi133=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f33200fa1b0 con 0x7f330c036ad0 2023-06-22T18:04:55.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.874+0000 7f33167fc700 1 -- 172.21.15.133:0/473842379 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f33200fa1b0 con 0x7f330c036ad0 2023-06-22T18:04:55.877 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mgr update... 2023-06-22T18:04:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f330c036ad0 msgr2=0x7f330c038f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f330c036ad0 0x7f330c038f80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f331c00b3b0 tx=0x7f331c009e30 comp rx=0 tx=0).stop 2023-06-22T18:04:55.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 msgr2=0x7f332010eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:04:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 0x7f332010eba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f331800bc30 tx=0x7f3318009820 comp rx=0 tx=0).stop 2023-06-22T18:04:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 shutdown_connections 2023-06-22T18:04:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f330c036ad0 0x7f330c038f80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f332010e700 0x7f332010eba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33200fc990 0x7f332010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 --2- 172.21.15.133:0/473842379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f332010d400 0x7f332010e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:04:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 >> 172.21.15.133:0/473842379 conn(0x7f33200f8050 msgr2=0x7f33200f9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:04:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 shutdown_connections 2023-06-22T18:04:55.882 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:04:55.876+0000 7f3328096700 1 -- 172.21.15.133:0/473842379 wait complete. 2023-06-22T18:04:56.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: Reconfiguring mon.c (monmap changed)... 2023-06-22T18:04:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:56.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: Reconfiguring daemon mon.c on smithi133 2023-06-22T18:04:56.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:56.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi101=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:56.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: Saving service mgr spec with placement smithi039=x;smithi101=y;smithi133=z;count:3 2023-06-22T18:04:56.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: Reconfiguring mon.c (monmap changed)... 2023-06-22T18:04:56.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:56.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:56.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:56.455 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: Reconfiguring daemon mon.c on smithi133 2023-06-22T18:04:56.455 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:56.455 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi101=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:56.455 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: Saving service mgr spec with placement smithi039=x;smithi101=y;smithi133=z;count:3 2023-06-22T18:04:56.455 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:56.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: Reconfiguring mon.c (monmap changed)... 2023-06-22T18:04:56.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-22T18:04:56.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-22T18:04:56.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:56.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: Reconfiguring daemon mon.c on smithi133 2023-06-22T18:04:56.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:56.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi039=x;smithi101=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:04:56.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: Saving service mgr spec with placement smithi039=x;smithi101=y;smithi133=z;count:3 2023-06-22T18:04:56.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:57.066 DEBUG:teuthology.orchestra.run.smithi101:mgr.y> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.y.service 2023-06-22T18:04:57.069 DEBUG:teuthology.orchestra.run.smithi133:mgr.z> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.z.service 2023-06-22T18:04:57.071 INFO:tasks.cephadm:Deploying OSDs... 2023-06-22T18:04:57.071 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-06-22T18:04:57.072 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2023-06-22T18:04:57.096 INFO:journalctl@ceph.mgr.z.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:04:57.100 INFO:journalctl@ceph.mgr.y.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:04:57.113 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-22T18:04:57.114 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2023-06-22T18:04:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-22T18:04:57.194 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-06-22 18:04:03.187743516 +0000 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-06-22 17:58:14.587283155 +0000 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-06-22 17:58:14.587283155 +0000 2023-06-22T18:04:57.195 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-06-22T18:04:57.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-22T18:04:57.299 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-06-22T18:04:57.300 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-06-22T18:04:57.300 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000400299 s, 1.3 MB/s 2023-06-22T18:04:57.302 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-22T18:04:57.342 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2023-06-22T18:04:57.420 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-22T18:04:57.420 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:57.420 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-06-22T18:04:57.420 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:57.420 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:57.421 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-06-22 18:04:03.187743516 +0000 2023-06-22T18:04:57.421 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-06-22 17:58:15.380266215 +0000 2023-06-22T18:04:57.421 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-06-22 17:58:15.380266215 +0000 2023-06-22T18:04:57.421 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-06-22T18:04:57.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-22T18:04:57.520 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-06-22T18:04:57.520 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-06-22T18:04:57.520 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000383197 s, 1.3 MB/s 2023-06-22T18:04:57.522 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-22T18:04:57.561 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2023-06-22T18:04:57.639 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-22T18:04:57.639 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:57.639 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-06-22T18:04:57.639 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:57.639 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:57.640 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-06-22 18:04:03.187743516 +0000 2023-06-22T18:04:57.640 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-06-22 17:58:16.159249575 +0000 2023-06-22T18:04:57.640 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-06-22 17:58:16.159249575 +0000 2023-06-22T18:04:57.640 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-06-22T18:04:57.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-22T18:04:57.741 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-06-22T18:04:57.741 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-06-22T18:04:57.741 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000338739 s, 1.5 MB/s 2023-06-22T18:04:57.743 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-22T18:04:57.781 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2023-06-22T18:04:57.858 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-22T18:04:57.859 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:57.859 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-06-22T18:04:57.859 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:57.859 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:57.859 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-06-22 18:04:03.187743516 +0000 2023-06-22T18:04:57.860 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-06-22 17:58:16.961232454 +0000 2023-06-22T18:04:57.860 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-06-22 17:58:16.961232454 +0000 2023-06-22T18:04:57.860 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2023-06-22T18:04:57.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-22T18:04:57.958 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-06-22T18:04:57.958 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-06-22T18:04:57.958 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000326065 s, 1.6 MB/s 2023-06-22T18:04:57.960 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-22T18:04:57.999 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-06-22T18:04:58.000 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/scratch_devs of=/dev/stdout 2023-06-22T18:04:58.017 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-22T18:04:58.018 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_1 2023-06-22T18:04:58.075 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-22T18:04:58.075 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:58.075 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 214540 Links: 1 2023-06-22T18:04:58.075 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:58.075 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:58.075 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-06-22 18:04:43.389016044 +0000 2023-06-22T18:04:58.076 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-06-22 17:49:04.221913694 +0000 2023-06-22T18:04:58.076 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-06-22 17:49:04.221913694 +0000 2023-06-22T18:04:58.076 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-06-22T18:04:58.076 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-22T18:04:58.140 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-06-22T18:04:58.141 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-06-22T18:04:58.141 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000264635 s, 1.9 MB/s 2023-06-22T18:04:58.142 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-22T18:04:58.199 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_2 2023-06-22T18:04:58.254 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-22T18:04:58.255 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:58.255 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 217164 Links: 1 2023-06-22T18:04:58.255 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:58.256 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:58.256 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-06-22 18:04:43.389016044 +0000 2023-06-22T18:04:58.256 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-06-22 17:49:04.745901435 +0000 2023-06-22T18:04:58.256 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-06-22 17:49:04.745901435 +0000 2023-06-22T18:04:58.256 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-06-22T18:04:58.257 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-22T18:04:58.319 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-06-22T18:04:58.319 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-06-22T18:04:58.319 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.00023324 s, 2.2 MB/s 2023-06-22T18:04:58.320 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-22T18:04:58.376 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_3 2023-06-22T18:04:58.431 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-22T18:04:58.431 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:58.431 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 214593 Links: 1 2023-06-22T18:04:58.431 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:58.431 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:58.432 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-06-22 18:04:43.389016044 +0000 2023-06-22T18:04:58.432 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-06-22 17:49:05.247889690 +0000 2023-06-22T18:04:58.432 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-06-22 17:49:05.247889690 +0000 2023-06-22T18:04:58.432 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-06-22T18:04:58.432 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-22T18:04:58.491 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.492 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.492 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:58.492 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:58.492 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:58.492 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:58.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:04:58.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-22T18:04:58.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:04:58.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:58.494 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-06-22T18:04:58.494 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-06-22T18:04:58.494 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000229889 s, 2.2 MB/s 2023-06-22T18:04:58.495 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-22T18:04:58.551 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_4 2023-06-22T18:04:58.605 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-22T18:04:58.605 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 214709 Links: 1 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-06-22 18:04:43.389016044 +0000 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-06-22 17:49:05.758877735 +0000 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-06-22 17:49:05.758877735 +0000 2023-06-22T18:04:58.606 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-06-22T18:04:58.607 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-22T18:04:58.668 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-06-22T18:04:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-06-22T18:04:58.669 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000210167 s, 2.4 MB/s 2023-06-22T18:04:58.670 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-22T18:04:58.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:04:58.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-22T18:04:58.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:04:58.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:58.725 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-06-22T18:04:58.725 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2023-06-22T18:04:58.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:04:58.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:04:58.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:04:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:04:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:04:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-22T18:04:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:04:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:04:58.740 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-22T18:04:58.741 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2023-06-22T18:04:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-22T18:04:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:58.797 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 210872 Links: 1 2023-06-22T18:04:58.797 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:58.798 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:58.798 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-06-22 18:04:34.517932749 +0000 2023-06-22T18:04:58.798 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-06-22 17:49:08.231346326 +0000 2023-06-22T18:04:58.798 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-06-22 17:49:08.231346326 +0000 2023-06-22T18:04:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-06-22T18:04:58.798 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-22T18:04:58.860 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-06-22T18:04:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-06-22T18:04:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000244144 s, 2.1 MB/s 2023-06-22T18:04:58.861 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-22T18:04:58.916 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2023-06-22T18:04:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-22T18:04:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:58.972 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 214699 Links: 1 2023-06-22T18:04:58.972 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:58.972 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-06-22 18:04:34.517932749 +0000 2023-06-22T18:04:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-06-22 17:49:08.760336548 +0000 2023-06-22T18:04:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-06-22 17:49:08.760336548 +0000 2023-06-22T18:04:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-06-22T18:04:58.973 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-22T18:04:59.037 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-06-22T18:04:59.038 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-06-22T18:04:59.038 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.00020457 s, 2.5 MB/s 2023-06-22T18:04:59.039 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-22T18:04:59.094 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2023-06-22T18:04:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-22T18:04:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:59.150 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 218244 Links: 1 2023-06-22T18:04:59.150 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:59.150 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:59.151 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-06-22 18:04:34.517932749 +0000 2023-06-22T18:04:59.151 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-06-22 17:49:09.260327300 +0000 2023-06-22T18:04:59.151 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-06-22 17:49:09.260327300 +0000 2023-06-22T18:04:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-06-22T18:04:59.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-22T18:04:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-06-22T18:04:59.212 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-06-22T18:04:59.213 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000201073 s, 2.5 MB/s 2023-06-22T18:04:59.214 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-22T18:04:59.269 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2023-06-22T18:04:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-22T18:04:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-22T18:04:59.323 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 213767 Links: 1 2023-06-22T18:04:59.323 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-22T18:04:59.323 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-06-22T18:04:59.323 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-06-22 18:04:34.517932749 +0000 2023-06-22T18:04:59.324 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-06-22 17:49:09.759318073 +0000 2023-06-22T18:04:59.324 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-06-22 17:49:09.759318073 +0000 2023-06-22T18:04:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-06-22T18:04:59.324 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-22T18:04:59.383 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:04:59 smithi133 ceph-mon[116867]: Deploying daemon mgr.z on smithi133 2023-06-22T18:04:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-06-22T18:04:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-06-22T18:04:59.385 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000189519 s, 2.7 MB/s 2023-06-22T18:04:59.386 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-22T18:04:59.441 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2023-06-22T18:04:59.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-22T18:04:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:04:59 smithi101 ceph-mon[116267]: Deploying daemon mgr.z on smithi133 2023-06-22T18:04:59.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:04:59 smithi039 ceph-mon[109976]: Deploying daemon mgr.z on smithi133 2023-06-22T18:04:59.985 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:04:59 smithi133 systemd[1]: Starting Ceph mgr.z for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:05:00.212 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:05:00.460 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mgr[119752]: -- 172.21.15.133:0/1569868939 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a2631c9c0 con 0x555a262f3000 2023-06-22T18:05:00.461 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:00 smithi133 bash[119689]: ae3a127bf62bb6702d4f9423a16ace067b1b3f60a19476e81db30bfc2049ba37 2023-06-22T18:05:00.734 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:00 smithi133 systemd[1]: Started Ceph mgr.z for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:05:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:05:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:00.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:00.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:00.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:00.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:00.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:05:00.891 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-22T18:05:00.891 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:05:00.891 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:01.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:05:01.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-22T18:05:01.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:05:01.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:01.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:01 smithi101 ceph-mon[116267]: Deploying daemon mgr.y on smithi101 2023-06-22T18:05:01.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:01 smithi039 ceph-mon[109976]: Deploying daemon mgr.y on smithi101 2023-06-22T18:05:01.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:01 smithi133 ceph-mon[116867]: Deploying daemon mgr.y on smithi101 2023-06-22T18:05:01.984 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:01 smithi133 conmon[119728]: 2023-06-22T18:05:01.573+0000 7f1cd33f61c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T18:05:01.985 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:01 smithi133 conmon[119728]: 2023-06-22T18:05:01.728+0000 7f1cd33f61c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T18:05:01.985 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:01 smithi133 conmon[119728]: 2023-06-22T18:05:01.813+0000 7f1cd33f61c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T18:05:02.209 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-22T18:05:02.210 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-22T18:05:02.210 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-06-22T18:05:02.210 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-06-22T18:05:02.210 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0292058 s, 359 MB/s 2023-06-22T18:05:02.210 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 2023-06-22T18:05:02.210 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-06-22T18:05:02.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:02 smithi101 systemd[1]: Starting Ceph mgr.y for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:05:02.379 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:02 smithi133 conmon[119728]: 2023-06-22T18:05:02.024+0000 7f1cd33f61c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:05:02.379 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:02 smithi133 conmon[119728]: 2023-06-22T18:05:02.219+0000 7f1cd33f61c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T18:05:02.661 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:02 smithi133 conmon[119728]: 2023-06-22T18:05:02.375+0000 7f1cd33f61c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T18:05:02.661 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:02 smithi133 conmon[119728]: 2023-06-22T18:05:02.454+0000 7f1cd33f61c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T18:05:02.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2023-06-22T18:05:02.781 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:02 smithi101 ceph-mon[116267]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:02.782 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:02 smithi101 ceph-mgr[117951]: -- 172.21.15.101:0/1955813439 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e38325a9c0 con 0x55e383231000 2023-06-22T18:05:02.782 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:02 smithi101 bash[117887]: 4366054d41acf10c4bfea08492732111642f4ca69ed43a582d80ddf8becd91ef 2023-06-22T18:05:02.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:02 smithi039 ceph-mon[109976]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:02.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:02 smithi133 ceph-mon[116867]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:03.102 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:02 smithi101 systemd[1]: Started Ceph mgr.y for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:05:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:05:03.484 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:03 smithi133 conmon[119728]: 2023-06-22T18:05:03.027+0000 7f1cd33f61c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T18:05:03.844 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:03 smithi133 conmon[119728]: 2023-06-22T18:05:03.537+0000 7f1cd33f61c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T18:05:03.844 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:03 smithi133 conmon[119728]: 2023-06-22T18:05:03.633+0000 7f1cd33f61c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T18:05:03.844 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:03 smithi133 conmon[119728]: 2023-06-22T18:05:03.731+0000 7f1cd33f61c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T18:05:04.030 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.030 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.030 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.030 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.030 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:04.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.103 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:03 smithi101 conmon[117927]: 2023-06-22T18:05:03.951+0000 7f9bc60251c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-22T18:05:04.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:04.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:04.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:04.425 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:04 smithi101 conmon[117927]: 2023-06-22T18:05:04.113+0000 7f9bc60251c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-22T18:05:04.426 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:04 smithi101 conmon[117927]: 2023-06-22T18:05:04.205+0000 7f9bc60251c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-22T18:05:04.426 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:04 smithi101 conmon[117927]: 2023-06-22T18:05:04.422+0000 7f9bc60251c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:05:04.609 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:04 smithi133 conmon[119728]: 2023-06-22T18:05:04.344+0000 7f1cd33f61c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T18:05:04.609 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:04 smithi133 conmon[119728]: 2023-06-22T18:05:04.516+0000 7f1cd33f61c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:05:04.797 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:04 smithi101 conmon[117927]: 2023-06-22T18:05:04.634+0000 7f9bc60251c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-22T18:05:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1694697367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fde10 msgr2=0x7f2a040fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1694697367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fde10 0x7f2a040fe260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f29f40049c0 tx=0x7f29f4009e30 comp rx=0 tx=0).stop 2023-06-22T18:05:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1694697367 shutdown_connections 2023-06-22T18:05:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1694697367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fe7a0 0x7f2a04105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1694697367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 0x7f2a040fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1694697367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fde10 0x7f2a040fe260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.866+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1694697367 >> 172.21.15.39:0/1694697367 conn(0x7f2a040f8320 msgr2=0x7f2a040fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:05:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.867+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1694697367 shutdown_connections 2023-06-22T18:05:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.867+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1694697367 wait complete. 2023-06-22T18:05:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.867+0000 7f2a0c49e700 1 Processor -- start 2023-06-22T18:05:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.867+0000 7f2a0c49e700 1 -- start start 2023-06-22T18:05:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0c49e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 0x7f2a0410fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0c49e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fde10 0x7f2a04110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0c49e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fe7a0 0x7f2a04117100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0c49e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04117710 con 0x7f2a040fde10 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0c49e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04117880 con 0x7f2a040fe7a0 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0c49e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04117a40 con 0x7f2a040fcc00 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0a23a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 0x7f2a0410fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0aa3b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fe7a0 0x7f2a04117100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a09a39700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fde10 0x7f2a04110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0a23a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 0x7f2a0410fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43992/0 (socket says 172.21.15.39:43992) 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a0a23a700 1 -- 172.21.15.39:0/1033965148 learned_addr learned my addr 172.21.15.39:0/1033965148 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a09a39700 1 -- 172.21.15.39:0/1033965148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 msgr2=0x7f2a0410fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a09a39700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 0x7f2a0410fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a09a39700 1 -- 172.21.15.39:0/1033965148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fe7a0 msgr2=0x7f2a04117100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a09a39700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fe7a0 0x7f2a04117100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:04.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.868+0000 7f2a09a39700 1 -- 172.21.15.39:0/1033965148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4005040 con 0x7f2a040fde10 2023-06-22T18:05:04.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.869+0000 7f2a09a39700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fde10 0x7f2a04110070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29f40049c0 tx=0x7f29f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:05:04.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.869+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f400e070 con 0x7f2a040fde10 2023-06-22T18:05:04.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.869+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29f4004160 con 0x7f2a040fde10 2023-06-22T18:05:04.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.869+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a04117cc0 con 0x7f2a040fde10 2023-06-22T18:05:04.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.869+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40123c0 con 0x7f2a040fde10 2023-06-22T18:05:04.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.869+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a04118170 con 0x7f2a040fde10 2023-06-22T18:05:04.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.870+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 13) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f29f4009500 con 0x7f2a040fde10 2023-06-22T18:05:04.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.870+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a04108440 con 0x7f2a040fde10 2023-06-22T18:05:04.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.871+0000 7f29faffd700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f29f0036890 0x7f29f0038d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:04.876 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.871+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f29f4049780 con 0x7f2a040fde10 2023-06-22T18:05:04.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.871+0000 7f2a0a23a700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f29f0036890 0x7f29f0038d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:04.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.872+0000 7f2a0a23a700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f29f0036890 0x7f29f0038d40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a00004ac0 tx=0x7f2a0000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:05:04.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:04.876+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f29f401c680 con 0x7f2a040fde10 2023-06-22T18:05:04.890 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:04 smithi133 conmon[119728]: 2023-06-22T18:05:04.605+0000 7f1cd33f61c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T18:05:04.891 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:04 smithi133 conmon[119728]: 2023-06-22T18:05:04.737+0000 7f1cd33f61c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T18:05:04.891 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:04 smithi133 conmon[119728]: 2023-06-22T18:05:04.830+0000 7f1cd33f61c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T18:05:05.094 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:05.091+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a04104730 con 0x7f29f0036890 2023-06-22T18:05:05.103 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:04 smithi101 conmon[117927]: 2023-06-22T18:05:04.794+0000 7f9bc60251c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-22T18:05:05.103 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:04 smithi101 conmon[117927]: 2023-06-22T18:05:04.873+0000 7f9bc60251c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-22T18:05:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:05:05.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:05:05.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:05:05.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:05:05.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:05.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:05.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:05.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-22T18:05:05.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-22T18:05:05.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:05.853 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:05 smithi101 conmon[117927]: 2023-06-22T18:05:05.443+0000 7f9bc60251c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-22T18:05:05.984 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:05 smithi133 conmon[119728]: 2023-06-22T18:05:05.691+0000 7f1cd33f61c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T18:05:05.984 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:05 smithi133 conmon[119728]: 2023-06-22T18:05:05.778+0000 7f1cd33f61c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T18:05:05.984 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:05 smithi133 conmon[119728]: 2023-06-22T18:05:05.863+0000 7f1cd33f61c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T18:05:05.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:05 smithi133 ceph-mon[116867]: Reconfiguring mgr.x (unknown last config time)... 2023-06-22T18:05:05.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:05 smithi133 ceph-mon[116867]: Reconfiguring daemon mgr.x on smithi039 2023-06-22T18:05:05.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:05 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:05.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:05 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:05.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:05 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:06.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:05 smithi039 ceph-mon[109976]: Reconfiguring mgr.x (unknown last config time)... 2023-06-22T18:05:06.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:05 smithi039 ceph-mon[109976]: Reconfiguring daemon mgr.x on smithi039 2023-06-22T18:05:06.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:05 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:06.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:05 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:06.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:05 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:06.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:05 smithi101 conmon[117927]: 2023-06-22T18:05:05.917+0000 7f9bc60251c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-22T18:05:06.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:06 smithi101 conmon[117927]: 2023-06-22T18:05:06.006+0000 7f9bc60251c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-22T18:05:06.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:06 smithi101 conmon[117927]: 2023-06-22T18:05:06.095+0000 7f9bc60251c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-22T18:05:06.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:05 smithi101 ceph-mon[116267]: Reconfiguring mgr.x (unknown last config time)... 2023-06-22T18:05:06.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:05 smithi101 ceph-mon[116267]: Reconfiguring daemon mgr.x on smithi039 2023-06-22T18:05:06.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:05 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:06.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:05 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:06.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:05 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:06.484 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:06 smithi133 conmon[119728]: 2023-06-22T18:05:06.009+0000 7f1cd33f61c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T18:05:06.484 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:06 smithi133 conmon[119728]: 2023-06-22T18:05:06.105+0000 7f1cd33f61c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T18:05:06.760 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:06 smithi133 conmon[119728]: 2023-06-22T18:05:06.497+0000 7f1cd33f61c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T18:05:06.761 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:06 smithi133 conmon[119728]: 2023-06-22T18:05:06.584+0000 7f1cd33f61c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T18:05:06.761 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:06 smithi133 conmon[119728]: 2023-06-22T18:05:06.671+0000 7f1cd33f61c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T18:05:06.930 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:06 smithi101 conmon[117927]: 2023-06-22T18:05:06.672+0000 7f9bc60251c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-22T18:05:06.930 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:06 smithi101 conmon[117927]: 2023-06-22T18:05:06.838+0000 7f9bc60251c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-22T18:05:06.930 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:06 smithi101 conmon[117927]: 2023-06-22T18:05:06.926+0000 7f9bc60251c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-22T18:05:06.930 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:06.931 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:06.932 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: Standby manager daemon z started 2023-06-22T18:05:06.932 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-22T18:05:06.932 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-22T18:05:06.932 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-22T18:05:06.932 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:06 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-22T18:05:07.117 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:07.117 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:07.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:07.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:07.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:07.118 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:07.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:07.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:07.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: Standby manager daemon z started 2023-06-22T18:05:07.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-22T18:05:07.119 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-22T18:05:07.120 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-22T18:05:07.120 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:06 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-22T18:05:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:07.215+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 14) v1 ==== 86210+0+0 (secure 0 0 0) 0x7f29f401adb0 con 0x7f2a040fde10 2023-06-22T18:05:07.234 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:05:06 smithi133 conmon[119728]: 2023-06-22T18:05:06.757+0000 7f1cd33f61c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T18:05:07.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:07.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:07.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:07.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:07.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:07.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: Standby manager daemon z started 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-22T18:05:07.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:06 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.133:0/1145896806' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-22T18:05:07.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:07 smithi101 conmon[117927]: 2023-06-22T18:05:07.052+0000 7f9bc60251c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-22T18:05:07.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:07 smithi101 conmon[117927]: 2023-06-22T18:05:07.140+0000 7f9bc60251c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-22T18:05:08.310 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:08 smithi101 ceph-mon[116267]: mgrmap e14: x(active, since 73s), standbys: z 2023-06-22T18:05:08.310 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-22T18:05:08.310 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:08 smithi101 ceph-mon[116267]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:08.311 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:07 smithi101 conmon[117927]: 2023-06-22T18:05:07.990+0000 7f9bc60251c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-22T18:05:08.311 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.075+0000 7f9bc60251c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-22T18:05:08.311 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.161+0000 7f9bc60251c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-22T18:05:08.311 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.307+0000 7f9bc60251c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-22T18:05:08.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:08 smithi133 ceph-mon[116867]: mgrmap e14: x(active, since 73s), standbys: z 2023-06-22T18:05:08.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-22T18:05:08.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:08 smithi133 ceph-mon[116867]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:08.603 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.403+0000 7f9bc60251c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-22T18:05:08.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:08 smithi039 ceph-mon[109976]: mgrmap e14: x(active, since 73s), standbys: z 2023-06-22T18:05:08.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-22T18:05:08.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:08 smithi039 ceph-mon[109976]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:09.049 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.787+0000 7f9bc60251c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-22T18:05:09.049 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.873+0000 7f9bc60251c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-22T18:05:09.049 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:08 smithi101 conmon[117927]: 2023-06-22T18:05:08.960+0000 7f9bc60251c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-22T18:05:09.353 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:05:09 smithi101 conmon[117927]: 2023-06-22T18:05:09.046+0000 7f9bc60251c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-22T18:05:09.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3671633720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f55d12a1-86b5-42a6-ad35-d5eaf3d99c62"}]: dispatch 2023-06-22T18:05:09.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3671633720' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f55d12a1-86b5-42a6-ad35-d5eaf3d99c62"}]': finished 2023-06-22T18:05:09.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: osdmap e5: 1 total, 0 up, 1 in 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: Standby manager daemon y started 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-22T18:05:09.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:09 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/226300340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:05:09.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3671633720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f55d12a1-86b5-42a6-ad35-d5eaf3d99c62"}]: dispatch 2023-06-22T18:05:09.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3671633720' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f55d12a1-86b5-42a6-ad35-d5eaf3d99c62"}]': finished 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: osdmap e5: 1 total, 0 up, 1 in 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: Standby manager daemon y started 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-22T18:05:09.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-22T18:05:09.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:09 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/226300340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:05:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:09.492+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 15) v1 ==== 129145+0+0 (secure 0 0 0) 0x7f29f40127f0 con 0x7f2a040fde10 2023-06-22T18:05:09.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3671633720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f55d12a1-86b5-42a6-ad35-d5eaf3d99c62"}]: dispatch 2023-06-22T18:05:09.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3671633720' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f55d12a1-86b5-42a6-ad35-d5eaf3d99c62"}]': finished 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: osdmap e5: 1 total, 0 up, 1 in 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: Standby manager daemon y started 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='mgr.? 172.21.15.101:0/992033872' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-22T18:05:09.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:09 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/226300340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:05:10.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:10 smithi101 ceph-mon[116267]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-06-22T18:05:10.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:10 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-22T18:05:10.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:10 smithi101 ceph-mon[116267]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:10 smithi039 ceph-mon[109976]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-06-22T18:05:10.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-22T18:05:10.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:10 smithi039 ceph-mon[109976]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:10.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:10 smithi133 ceph-mon[116867]: mgrmap e15: x(active, since 75s), standbys: z, y 2023-06-22T18:05:10.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:10 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-22T18:05:10.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:10 smithi133 ceph-mon[116867]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:12.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:12 smithi039 ceph-mon[109976]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:12.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:12 smithi133 ceph-mon[116867]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:13.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:12 smithi101 ceph-mon[116267]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:14.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:14 smithi039 ceph-mon[109976]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:14.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:14 smithi133 ceph-mon[116867]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:15.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:14 smithi101 ceph-mon[116267]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:16 smithi039 ceph-mon[109976]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-22T18:05:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:16 smithi039 ceph-mon[109976]: Deploying daemon osd.0 on smithi039 2023-06-22T18:05:16.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:16 smithi133 ceph-mon[116867]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:16.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-22T18:05:16.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:16.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:16 smithi133 ceph-mon[116867]: Deploying daemon osd.0 on smithi039 2023-06-22T18:05:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:16 smithi101 ceph-mon[116267]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-22T18:05:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:16 smithi101 ceph-mon[116267]: Deploying daemon osd.0 on smithi039 2023-06-22T18:05:18.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:18 smithi133 ceph-mon[116867]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:19.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:18 smithi039 ceph-mon[109976]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:19.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:18 smithi101 ceph-mon[116267]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:20.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:20 smithi133 ceph-mon[116867]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:21.032 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:20 smithi039 ceph-mon[109976]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:21.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:20 smithi101 ceph-mon[116267]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:22.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:22 smithi133 ceph-mon[116867]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:22.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:22.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:22.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:22 smithi101 ceph-mon[116267]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:22 smithi039 ceph-mon[109976]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:24.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:24 smithi133 ceph-mon[116867]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:24 smithi101 ceph-mon[116267]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:25.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:24 smithi039 ceph-mon[109976]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:26.008 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.005+0000 7f29faffd700 1 -- 172.21.15.39:0/1033965148 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2a04104730 con 0x7f29f0036890 2023-06-22T18:05:26.008 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2023-06-22T18:05:26.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f29f0036890 msgr2=0x7f29f0038d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:26.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f29f0036890 0x7f29f0038d40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a00004ac0 tx=0x7f2a0000b040 comp rx=0 tx=0).stop 2023-06-22T18:05:26.012 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fde10 msgr2=0x7f2a04110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fde10 0x7f2a04110070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29f40049c0 tx=0x7f29f4009e30 comp rx=0 tx=0).stop 2023-06-22T18:05:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 shutdown_connections 2023-06-22T18:05:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f29f0036890 0x7f29f0038d40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a040fe7a0 0x7f2a04117100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a040fcc00 0x7f2a0410fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 --2- 172.21.15.39:0/1033965148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040fde10 0x7f2a04110070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 >> 172.21.15.39:0/1033965148 conn(0x7f2a040f8320 msgr2=0x7f2a040fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:05:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 shutdown_connections 2023-06-22T18:05:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:05:26.009+0000 7f2a0c49e700 1 -- 172.21.15.39:0/1033965148 wait complete. 2023-06-22T18:05:26.185 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:26.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:26.186 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:26.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:26.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:26.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:26.441 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.0.service 2023-06-22T18:05:26.445 INFO:tasks.cephadm:Deploying osd.1 on smithi101 with /dev/vg_nvme/lv_4... 2023-06-22T18:05:26.446 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-22T18:05:26.510 INFO:journalctl@ceph.osd.0.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:05:27.030 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:05:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:26 smithi133 ceph-mon[116867]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:26 smithi133 ceph-mon[116867]: from='osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-22T18:05:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:26 smithi133 ceph-mon[116867]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-22T18:05:27.240 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:26 smithi101 ceph-mon[116267]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:27.240 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:27.240 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:27.240 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:27.241 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:26 smithi101 ceph-mon[116267]: from='osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-22T18:05:27.241 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:26 smithi101 ceph-mon[116267]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-22T18:05:27.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:26 smithi039 ceph-mon[109976]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:27.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:27.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:27.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:27.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:26 smithi039 ceph-mon[109976]: from='osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-22T18:05:27.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:26 smithi039 ceph-mon[109976]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-22T18:05:28.131 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:27 smithi039 ceph-mon[109976]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-22T18:05:28.131 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:27 smithi039 ceph-mon[109976]: osdmap e6: 1 total, 0 up, 1 in 2023-06-22T18:05:28.132 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:27 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:28.132 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:27 smithi039 ceph-mon[109976]: from='osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:05:28.132 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:27 smithi039 ceph-mon[109976]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:05:28.132 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:05:27 smithi039 conmon[126227]: 2023-06-22T18:05:27.981+0000 7fdb104b0700 -1 osd.0 0 waiting for initial osdmap 2023-06-22T18:05:28.132 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:05:27 smithi039 conmon[126227]: 2023-06-22T18:05:27.985+0000 7fdb0b2ab700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:05:28.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:27 smithi133 ceph-mon[116867]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-22T18:05:28.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:27 smithi133 ceph-mon[116867]: osdmap e6: 1 total, 0 up, 1 in 2023-06-22T18:05:28.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:27 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:28.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:27 smithi133 ceph-mon[116867]: from='osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:05:28.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:27 smithi133 ceph-mon[116867]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:05:28.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:27 smithi101 ceph-mon[116267]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-22T18:05:28.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:27 smithi101 ceph-mon[116267]: osdmap e6: 1 total, 0 up, 1 in 2023-06-22T18:05:28.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:27 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:28.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:27 smithi101 ceph-mon[116267]: from='osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:05:28.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:27 smithi101 ceph-mon[116267]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:05:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-22T18:05:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-22T18:05:28.892 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-06-22T18:05:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-06-22T18:05:28.892 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108796 s, 964 MB/s 2023-06-22T18:05:28.892 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-06-22T18:05:29.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:28 smithi039 ceph-mon[109976]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:29.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:28 smithi039 ceph-mon[109976]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:05:29.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:28 smithi039 ceph-mon[109976]: osdmap e7: 1 total, 0 up, 1 in 2023-06-22T18:05:29.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:28 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:29.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:28 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:29.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:28 smithi101 ceph-mon[116267]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:29.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:28 smithi101 ceph-mon[116267]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:05:29.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:28 smithi101 ceph-mon[116267]: osdmap e7: 1 total, 0 up, 1 in 2023-06-22T18:05:29.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:28 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:29.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:28 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:29.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:28 smithi133 ceph-mon[116867]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:29.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:28 smithi133 ceph-mon[116867]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:05:29.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:28 smithi133 ceph-mon[116867]: osdmap e7: 1 total, 0 up, 1 in 2023-06-22T18:05:29.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:28 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:29.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:28 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:29.558 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_4 2023-06-22T18:05:30.134 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:05:30.306 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:05:30.307 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:05:30.307 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:30.307 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770] boot 2023-06-22T18:05:30.307 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: osdmap e8: 1 total, 1 up, 1 in 2023-06-22T18:05:30.308 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:30.308 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.308 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.309 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:05:30.309 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.309 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:30.309 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:30.310 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:05:30.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:05:30.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:30.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770] boot 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: osdmap e8: 1 total, 1 up, 1 in 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:30.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:30.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:05:30.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:05:30.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:30.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: osd.0 [v2:172.21.15.39:6802/471291770,v1:172.21.15.39:6803/471291770] boot 2023-06-22T18:05:30.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: osdmap e8: 1 total, 1 up, 1 in 2023-06-22T18:05:30.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-22T18:05:30.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:05:30.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:30.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:30.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:30.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:31.300 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:30 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi039 2023-06-22T18:05:31.300 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:30 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi039 to 17267M 2023-06-22T18:05:31.300 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:30 smithi101 ceph-mon[116267]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:31.300 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:30 smithi101 ceph-mon[116267]: osdmap e9: 1 total, 1 up, 1 in 2023-06-22T18:05:31.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:30 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi039 2023-06-22T18:05:31.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:30 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi039 to 17267M 2023-06-22T18:05:31.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:30 smithi039 ceph-mon[109976]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:31.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:30 smithi039 ceph-mon[109976]: osdmap e9: 1 total, 1 up, 1 in 2023-06-22T18:05:31.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:30 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi039 2023-06-22T18:05:31.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:30 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi039 to 17267M 2023-06-22T18:05:31.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:30 smithi133 ceph-mon[116867]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-22T18:05:31.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:30 smithi133 ceph-mon[116867]: osdmap e9: 1 total, 1 up, 1 in 2023-06-22T18:05:31.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- 172.21.15.101:0/3857749577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0102880 msgr2=0x7f8dc0104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:31.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/3857749577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0102880 0x7f8dc0104d30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009d60 tx=0x7f8db8004d10 comp rx=0 tx=0).stop 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- 172.21.15.101:0/3857749577 shutdown_connections 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/3857749577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0102880 0x7f8dc0104d30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/3857749577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dc0066050 0x7f8dc00ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/3857749577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc00ffde0 0x7f8dc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- 172.21.15.101:0/3857749577 >> 172.21.15.101:0/3857749577 conn(0x7f8dc00f7210 msgr2=0x7f8dc00f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- 172.21.15.101:0/3857749577 shutdown_connections 2023-06-22T18:05:31.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- 172.21.15.101:0/3857749577 wait complete. 2023-06-22T18:05:31.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 Processor -- start 2023-06-22T18:05:31.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- start start 2023-06-22T18:05:31.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 0x7f8dc010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc00ffde0 0x7f8dc010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dc0102880 0x7f8dc0114e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0115490 con 0x7f8dc0102880 2023-06-22T18:05:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0115600 con 0x7f8dc0066050 2023-06-22T18:05:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.707+0000 7f8dc5a46700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc01157c0 con 0x7f8dc00ffde0 2023-06-22T18:05:31.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 0x7f8dc010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 0x7f8dc010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44020/0 (socket says 172.21.15.101:44020) 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 -- 172.21.15.101:0/2264342960 learned_addr learned my addr 172.21.15.101:0/2264342960 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbe7fc700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc00ffde0 0x7f8dc010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbf7fe700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dc0102880 0x7f8dc0114e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 -- 172.21.15.101:0/2264342960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc00ffde0 msgr2=0x7f8dc010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc00ffde0 0x7f8dc010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:31.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 -- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dc0102880 msgr2=0x7f8dc0114e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dc0102880 0x7f8dc0114e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 -- 172.21.15.101:0/2264342960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db8009940 con 0x7f8dc0066050 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dbeffd700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 0x7f8dc010d8b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b810 tx=0x7f8db0009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000f5f0 con 0x7f8dc0066050 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db000fc20 con 0x7f8dc0066050 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc0115aa0 con 0x7f8dc0066050 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0010cc0 con 0x7f8dc0066050 2023-06-22T18:05:31.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.708+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc0115f90 con 0x7f8dc0066050 2023-06-22T18:05:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.709+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc00f8d40 con 0x7f8dc0066050 2023-06-22T18:05:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.710+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 15) v1 ==== 129145+0+0 (secure 0 0 0) 0x7f8db000fd90 con 0x7f8dc0066050 2023-06-22T18:05:31.718 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.711+0000 7f8da77fe700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8da809b670 0x7f8da809db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:31.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.711+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1763+0+0 (secure 0 0 0) 0x7f8db00c3bc0 con 0x7f8dc0066050 2023-06-22T18:05:31.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.711+0000 7f8dbe7fc700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8da809b670 0x7f8da809db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:31.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.712+0000 7f8dbe7fc700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8da809b670 0x7f8da809db20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006fd0 tx=0x7f8db4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:05:31.719 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.714+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f8db0090df0 con 0x7f8dc0066050 2023-06-22T18:05:31.908 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:31.904+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dc005de70 con 0x7f8da809b670 2023-06-22T18:05:32.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:32.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:32.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:32.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:32.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:32.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:32.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:32.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:32.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:33.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:33 smithi101 ceph-mon[116267]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:33.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:33 smithi101 ceph-mon[116267]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:33.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:33 smithi039 ceph-mon[109976]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:33.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:33 smithi039 ceph-mon[109976]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:33.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:33 smithi133 ceph-mon[116867]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:33.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:33 smithi133 ceph-mon[116867]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:35.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:35 smithi101 ceph-mon[116267]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:35.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:35 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/2712897229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]: dispatch 2023-06-22T18:05:35.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:35 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]: dispatch 2023-06-22T18:05:35.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:35 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]': finished 2023-06-22T18:05:35.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:35 smithi101 ceph-mon[116267]: osdmap e10: 2 total, 1 up, 2 in 2023-06-22T18:05:35.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:35.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:35 smithi039 ceph-mon[109976]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:35 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/2712897229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]: dispatch 2023-06-22T18:05:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:35 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]: dispatch 2023-06-22T18:05:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:35 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]': finished 2023-06-22T18:05:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:35 smithi039 ceph-mon[109976]: osdmap e10: 2 total, 1 up, 2 in 2023-06-22T18:05:35.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:35.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:35 smithi133 ceph-mon[116867]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:35.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:35 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/2712897229' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]: dispatch 2023-06-22T18:05:35.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:35 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]: dispatch 2023-06-22T18:05:35.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:35 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d8fb427-c4b2-45c3-ac42-dea08c42e4b6"}]': finished 2023-06-22T18:05:35.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:35 smithi133 ceph-mon[116867]: osdmap e10: 2 total, 1 up, 2 in 2023-06-22T18:05:35.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:36.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:36 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/73799987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:05:36.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:36 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/73799987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:05:36.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:36 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/73799987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:05:37.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:37 smithi101 ceph-mon[116267]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:37.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:37 smithi039 ceph-mon[109976]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:37.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:37 smithi133 ceph-mon[116867]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:39.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:39 smithi101 ceph-mon[116267]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:39.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:39 smithi039 ceph-mon[109976]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:39.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:39 smithi133 ceph-mon[116867]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:41.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:41 smithi101 ceph-mon[116267]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:41.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:41 smithi039 ceph-mon[109976]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:41.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:41 smithi133 ceph-mon[116867]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:42.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-22T18:05:42.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:42.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-22T18:05:42.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:42.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-22T18:05:42.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:43 smithi101 ceph-mon[116267]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:43 smithi101 ceph-mon[116267]: Deploying daemon osd.1 on smithi101 2023-06-22T18:05:43.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:43 smithi039 ceph-mon[109976]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:43.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:43 smithi039 ceph-mon[109976]: Deploying daemon osd.1 on smithi101 2023-06-22T18:05:43.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:43 smithi133 ceph-mon[116867]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:43.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:43 smithi133 ceph-mon[116867]: Deploying daemon osd.1 on smithi101 2023-06-22T18:05:45.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:45 smithi101 ceph-mon[116267]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:45.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:45 smithi039 ceph-mon[109976]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:45 smithi133 ceph-mon[116867]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:47.337 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:47 smithi101 ceph-mon[116267]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:47.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:47 smithi039 ceph-mon[109976]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:47.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:47 smithi133 ceph-mon[116867]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:48.295 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:48.295 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:48.296 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:48.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:48.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:48.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:48.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:48.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:48.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:49.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:49 smithi101 ceph-mon[116267]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:49.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:49 smithi039 ceph-mon[109976]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:49.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:49 smithi133 ceph-mon[116867]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:50.980 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.977+0000 7f8da77fe700 1 -- 172.21.15.101:0/2264342960 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8dc005de70 con 0x7f8da809b670 2023-06-22T18:05:50.981 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 1 on host 'smithi101' 2023-06-22T18:05:50.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8da809b670 msgr2=0x7f8da809db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:50.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8da809b670 0x7f8da809db20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006fd0 tx=0x7f8db4008040 comp rx=0 tx=0).stop 2023-06-22T18:05:50.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 msgr2=0x7f8dc010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:50.982 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 0x7f8dc010d8b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8db000b810 tx=0x7f8db0009220 comp rx=0 tx=0).stop 2023-06-22T18:05:50.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 shutdown_connections 2023-06-22T18:05:50.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8da809b670 0x7f8da809db20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:50.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dc0102880 0x7f8dc0114e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:50.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8dc0066050 0x7f8dc010d8b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:50.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 --2- 172.21.15.101:0/2264342960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dc00ffde0 0x7f8dc010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:50.983 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 >> 172.21.15.101:0/2264342960 conn(0x7f8dc00f7210 msgr2=0x7f8dc00fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:05:50.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 shutdown_connections 2023-06-22T18:05:50.984 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:05:50.979+0000 7f8dc5a46700 1 -- 172.21.15.101:0/2264342960 wait complete. 2023-06-22T18:05:51.171 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:50 smithi101 ceph-mon[116267]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:51.172 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.172 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.172 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:51.172 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:51.172 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:50 smithi039 ceph-mon[109976]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:51.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:51.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:51.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:50 smithi133 ceph-mon[116867]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:51.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:51.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.688 DEBUG:teuthology.orchestra.run.smithi101:osd.1> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.1.service 2023-06-22T18:05:51.690 INFO:tasks.cephadm:Deploying osd.2 on smithi133 with /dev/vg_nvme/lv_4... 2023-06-22T18:05:51.690 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-22T18:05:51.716 INFO:journalctl@ceph.osd.1.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:05:51.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:51.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:51.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:51 smithi133 ceph-mon[116867]: from='osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-22T18:05:51.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:51 smithi133 ceph-mon[116867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-22T18:05:52.211 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:52.212 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:52.212 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:52.212 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:51 smithi101 ceph-mon[116267]: from='osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-22T18:05:52.212 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:51 smithi101 ceph-mon[116267]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-22T18:05:52.277 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:05:52.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:05:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:51 smithi039 ceph-mon[109976]: from='osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-22T18:05:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:51 smithi039 ceph-mon[109976]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-22T18:05:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:52 smithi133 ceph-mon[116867]: pgmap v59: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:52 smithi133 ceph-mon[116867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-22T18:05:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:52 smithi133 ceph-mon[116867]: osdmap e11: 2 total, 1 up, 2 in 2023-06-22T18:05:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:52 smithi133 ceph-mon[116867]: from='osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:05:53.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:52 smithi133 ceph-mon[116867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:05:53.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:52 smithi101 ceph-mon[116267]: pgmap v59: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:53.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:52 smithi101 ceph-mon[116267]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-22T18:05:53.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:52 smithi101 ceph-mon[116267]: osdmap e11: 2 total, 1 up, 2 in 2023-06-22T18:05:53.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:53.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:52 smithi101 ceph-mon[116267]: from='osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:05:53.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:52 smithi101 ceph-mon[116267]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:05:53.313 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:05:52 smithi101 conmon[122684]: 2023-06-22T18:05:52.971+0000 7fcf11b05700 -1 osd.1 0 waiting for initial osdmap 2023-06-22T18:05:53.313 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:05:52 smithi101 conmon[122684]: 2023-06-22T18:05:52.974+0000 7fcf0e904700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:05:53.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:52 smithi039 ceph-mon[109976]: pgmap v59: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:52 smithi039 ceph-mon[109976]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-22T18:05:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:52 smithi039 ceph-mon[109976]: osdmap e11: 2 total, 1 up, 2 in 2023-06-22T18:05:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:52 smithi039 ceph-mon[109976]: from='osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:05:53.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:52 smithi039 ceph-mon[109976]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:05:54.000 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-22T18:05:54.001 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-22T18:05:54.001 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-06-22T18:05:54.001 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-06-22T18:05:54.001 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110477 s, 949 MB/s 2023-06-22T18:05:54.001 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-06-22T18:05:54.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:53 smithi133 ceph-mon[116867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:05:54.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:53 smithi133 ceph-mon[116867]: osdmap e12: 2 total, 1 up, 2 in 2023-06-22T18:05:54.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:53 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:54.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:53 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:54.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:53 smithi101 ceph-mon[116267]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:05:54.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:53 smithi101 ceph-mon[116267]: osdmap e12: 2 total, 1 up, 2 in 2023-06-22T18:05:54.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:53 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:54.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:53 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:54.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:53 smithi039 ceph-mon[109976]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:05:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:53 smithi039 ceph-mon[109976]: osdmap e12: 2 total, 1 up, 2 in 2023-06-22T18:05:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:54.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:54.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2023-06-22T18:05:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:05:55.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:05:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:05:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: pgmap v62: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728] boot 2023-06-22T18:05:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: osdmap e13: 2 total, 2 up, 2 in 2023-06-22T18:05:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:05:55.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:55.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:55.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: pgmap v62: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728] boot 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: osdmap e13: 2 total, 2 up, 2 in 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:05:55.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:55.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:55.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:05:55.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:05:55.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: pgmap v62: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-06-22T18:05:55.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: osd.1 [v2:172.21.15.101:6800/840111728,v1:172.21.15.101:6801/840111728] boot 2023-06-22T18:05:55.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: osdmap e13: 2 total, 2 up, 2 in 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:55.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:05:55.455 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:05:56.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:55 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi101 2023-06-22T18:05:56.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:55 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi101 to 17172M 2023-06-22T18:05:56.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:55 smithi133 ceph-mon[116867]: osdmap e14: 2 total, 2 up, 2 in 2023-06-22T18:05:56.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:55 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi101 2023-06-22T18:05:56.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:55 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi101 to 17172M 2023-06-22T18:05:56.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:55 smithi101 ceph-mon[116267]: osdmap e14: 2 total, 2 up, 2 in 2023-06-22T18:05:56.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:55 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi101 2023-06-22T18:05:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:55 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi101 to 17172M 2023-06-22T18:05:56.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:55 smithi039 ceph-mon[109976]: osdmap e14: 2 total, 2 up, 2 in 2023-06-22T18:05:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 -- 172.21.15.133:0/3470378747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fe820 msgr2=0x7fc0101050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 --2- 172.21.15.133:0/3470378747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fe820 0x7fc0101050e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc008009d60 tx=0x7fc008004d10 comp rx=0 tx=0).stop 2023-06-22T18:05:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 -- 172.21.15.133:0/3470378747 shutdown_connections 2023-06-22T18:05:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 --2- 172.21.15.133:0/3470378747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fe820 0x7fc0101050e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 --2- 172.21.15.133:0/3470378747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fcc80 0x7fc0100fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 --2- 172.21.15.133:0/3470378747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fde90 0x7fc0100fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 -- 172.21.15.133:0/3470378747 >> 172.21.15.133:0/3470378747 conn(0x7fc0100f83c0 msgr2=0x7fc0100fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:05:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 -- 172.21.15.133:0/3470378747 shutdown_connections 2023-06-22T18:05:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.701+0000 7fc016a1d700 1 -- 172.21.15.133:0/3470378747 wait complete. 2023-06-22T18:05:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 Processor -- start 2023-06-22T18:05:56.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 -- start start 2023-06-22T18:05:56.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 0x7fc010071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:56.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fde90 0x7fc01006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:56.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fe820 0x7fc0100702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:56.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010070a30 con 0x7fc0100fe820 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010070ba0 con 0x7fc0100fde90 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc016a1d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc010117930 con 0x7fc0100fcc80 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 0x7fc010071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 0x7fc010071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47486/0 (socket says 172.21.15.133:47486) 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc00ffff700 1 -- 172.21.15.133:0/1620319493 learned_addr learned my addr 172.21.15.133:0/1620319493 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.702+0000 7fc00f7fe700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fde90 0x7fc01006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:56.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00ffff700 1 -- 172.21.15.133:0/1620319493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fde90 msgr2=0x7fc01006fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00ffff700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fde90 0x7fc01006fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc014fba700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fe820 0x7fc0100702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00ffff700 1 -- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fe820 msgr2=0x7fc0100702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00ffff700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fe820 0x7fc0100702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00ffff700 1 -- 172.21.15.133:0/1620319493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc008009940 con 0x7fc0100fcc80 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00f7fe700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fde90 0x7fc01006fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc014fba700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fe820 0x7fc0100702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-22T18:05:56.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00ffff700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 0x7fc010071720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b810 tx=0x7fc000009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000f5f0 con 0x7fc0100fcc80 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc010117be0 con 0x7fc0100fcc80 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc00000fc20 con 0x7fc0100fcc80 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000010cc0 con 0x7fc0100fcc80 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.703+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0101180d0 con 0x7fc0100fcc80 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.704+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010108520 con 0x7fc0100fcc80 2023-06-22T18:05:56.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.705+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) v1 ==== 129145+0+0 (secure 0 0 0) 0x7fc00000fd90 con 0x7fc0100fcc80 2023-06-22T18:05:56.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.705+0000 7fc00cff9700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fbff809b670 0x7fbff809db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:05:56.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.706+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2305+0+0 (secure 0 0 0) 0x7fc0000c3e40 con 0x7fc0100fcc80 2023-06-22T18:05:56.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.706+0000 7fc00f7fe700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fbff809b670 0x7fbff809db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:05:56.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.706+0000 7fc00f7fe700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fbff809b670 0x7fbff809db20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc004006fd0 tx=0x7fc004008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:05:56.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.709+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc000090e50 con 0x7fc0100fcc80 2023-06-22T18:05:56.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:05:56.897+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc01005de70 con 0x7fbff809b670 2023-06-22T18:05:57.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:56 smithi101 ceph-mon[116267]: pgmap v65: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:05:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:57.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:56 smithi039 ceph-mon[109976]: pgmap v65: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:05:57.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:57.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:57.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:57.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:56 smithi133 ceph-mon[116867]: pgmap v65: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:05:57.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:05:57.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:05:57.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:05:58.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:58 smithi101 ceph-mon[116267]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:58.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:58 smithi039 ceph-mon[109976]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:58.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:58 smithi133 ceph-mon[116867]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:05:59.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:05:59 smithi133 ceph-mon[116867]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:05:59.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:05:59 smithi101 ceph-mon[116267]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:05:59.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:05:59 smithi039 ceph-mon[109976]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:00.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:00 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/2888399554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]: dispatch 2023-06-22T18:06:00.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:00 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]: dispatch 2023-06-22T18:06:00.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:00 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]': finished 2023-06-22T18:06:00.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:00 smithi101 ceph-mon[116267]: osdmap e15: 3 total, 2 up, 3 in 2023-06-22T18:06:00.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:00.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:00 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/2888399554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]: dispatch 2023-06-22T18:06:00.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:00 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]: dispatch 2023-06-22T18:06:00.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:00 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]': finished 2023-06-22T18:06:00.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:00 smithi039 ceph-mon[109976]: osdmap e15: 3 total, 2 up, 3 in 2023-06-22T18:06:00.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:00.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:00 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/2888399554' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]: dispatch 2023-06-22T18:06:00.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:00 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]: dispatch 2023-06-22T18:06:00.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:00 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c714cec-14fc-4aed-b3f1-f0773edd197c"}]': finished 2023-06-22T18:06:00.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:00 smithi133 ceph-mon[116867]: osdmap e15: 3 total, 2 up, 3 in 2023-06-22T18:06:00.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:01.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:01 smithi101 ceph-mon[116267]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:01.398 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:01 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/696234205' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:01.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:01 smithi039 ceph-mon[109976]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:01.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:01 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/696234205' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:01.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:01 smithi133 ceph-mon[116867]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:01.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:01 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/696234205' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:03.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:03 smithi101 ceph-mon[116267]: pgmap v69: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:03.411 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:03 smithi133 ceph-mon[116867]: pgmap v69: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:03.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:03 smithi039 ceph-mon[109976]: pgmap v69: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:05.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:05 smithi101 ceph-mon[116267]: pgmap v70: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:05.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:05 smithi039 ceph-mon[109976]: pgmap v70: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:05.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:05 smithi133 ceph-mon[116867]: pgmap v70: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:07.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:07 smithi133 ceph-mon[116867]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:07.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:07 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-22T18:06:07.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:07 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:07.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:07 smithi101 ceph-mon[116267]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:07.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:07 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-22T18:06:07.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:07 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:07 smithi039 ceph-mon[109976]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:07.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:07 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-22T18:06:07.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:07 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:08.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:08 smithi101 ceph-mon[116267]: Deploying daemon osd.2 on smithi133 2023-06-22T18:06:08.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:08 smithi039 ceph-mon[109976]: Deploying daemon osd.2 on smithi133 2023-06-22T18:06:08.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:08 smithi133 ceph-mon[116867]: Deploying daemon osd.2 on smithi133 2023-06-22T18:06:09.125 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:09 smithi133 ceph-mon[116867]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:09.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:09 smithi101 ceph-mon[116267]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:09.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:09 smithi039 ceph-mon[109976]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:11.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:11 smithi101 ceph-mon[116267]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:11.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:11 smithi039 ceph-mon[109976]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:11.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:11 smithi133 ceph-mon[116867]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:13.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:13 smithi101 ceph-mon[116267]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:13.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:13 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:13.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:13 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:13.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:13 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:13.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:13 smithi133 ceph-mon[116867]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:13.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:13 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:13.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:13 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:13.420 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:13 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:13.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:13 smithi039 ceph-mon[109976]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:13.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:13 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:13.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:13 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:13.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:13 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:15.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:15 smithi101 ceph-mon[116267]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:15.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:15 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:15 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:15 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:15.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:15 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:15.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:15 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:15 smithi039 ceph-mon[109976]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:15.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:15 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:15 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:15 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:15.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:15 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:15.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:15 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:15 smithi133 ceph-mon[116867]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:15.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:15 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:15 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:15 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:15.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:15 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:15.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:15 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:15.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.820+0000 7fc00cff9700 1 -- 172.21.15.133:0/1620319493 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc01005de70 con 0x7fbff809b670 2023-06-22T18:06:15.824 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 2 on host 'smithi133' 2023-06-22T18:06:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fbff809b670 msgr2=0x7fbff809db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fbff809b670 0x7fbff809db20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc004006fd0 tx=0x7fc004008040 comp rx=0 tx=0).stop 2023-06-22T18:06:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 msgr2=0x7fc010071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 0x7fc010071720 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc00000b810 tx=0x7fc000009220 comp rx=0 tx=0).stop 2023-06-22T18:06:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 shutdown_connections 2023-06-22T18:06:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fbff809b670 0x7fbff809db20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0100fe820 0x7fc0100702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0100fcc80 0x7fc010071720 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 --2- 172.21.15.133:0/1620319493 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0100fde90 0x7fc01006fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:15.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 >> 172.21.15.133:0/1620319493 conn(0x7fc0100f83c0 msgr2=0x7fc0100fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:06:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 shutdown_connections 2023-06-22T18:06:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:06:15.823+0000 7fc016a1d700 1 -- 172.21.15.133:0/1620319493 wait complete. 2023-06-22T18:06:16.540 DEBUG:teuthology.orchestra.run.smithi133:osd.2> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.2.service 2023-06-22T18:06:16.542 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2023-06-22T18:06:16.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-22T18:06:16.571 INFO:journalctl@ceph.osd.2.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:06:16.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:16 smithi133 ceph-mon[116867]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:16.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:16.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:16.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:16.875 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:16 smithi133 ceph-mon[116867]: from='osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-22T18:06:16.875 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:16 smithi133 ceph-mon[116867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-22T18:06:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:16 smithi039 ceph-mon[109976]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:16 smithi039 ceph-mon[109976]: from='osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-22T18:06:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:16 smithi039 ceph-mon[109976]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-22T18:06:17.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:16 smithi101 ceph-mon[116267]: pgmap v76: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:16 smithi101 ceph-mon[116267]: from='osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-22T18:06:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:16 smithi101 ceph-mon[116267]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-22T18:06:17.319 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:06:18.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:18 smithi039 ceph-mon[109976]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-22T18:06:18.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:18 smithi039 ceph-mon[109976]: osdmap e16: 3 total, 2 up, 3 in 2023-06-22T18:06:18.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:18 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:18.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:18 smithi039 ceph-mon[109976]: from='osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:06:18.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:18 smithi039 ceph-mon[109976]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:06:18.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:18 smithi101 ceph-mon[116267]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-22T18:06:18.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:18 smithi101 ceph-mon[116267]: osdmap e16: 3 total, 2 up, 3 in 2023-06-22T18:06:18.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:18 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:18.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:18 smithi101 ceph-mon[116267]: from='osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:06:18.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:18 smithi101 ceph-mon[116267]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:06:18.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:18 smithi133 ceph-mon[116867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-22T18:06:18.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:18 smithi133 ceph-mon[116867]: osdmap e16: 3 total, 2 up, 3 in 2023-06-22T18:06:18.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:18 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:18.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:18 smithi133 ceph-mon[116867]: from='osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:06:18.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:18 smithi133 ceph-mon[116867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:06:18.486 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:18 smithi133 conmon[124549]: 2023-06-22T18:06:18.066+0000 7f504036d700 -1 osd.2 0 waiting for initial osdmap 2023-06-22T18:06:18.486 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:18 smithi133 conmon[124549]: 2023-06-22T18:06:18.070+0000 7f503c96b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:06:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-22T18:06:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-22T18:06:19.214 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-06-22T18:06:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-06-22T18:06:19.215 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0291312 s, 360 MB/s 2023-06-22T18:06:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-06-22T18:06:19.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:19 smithi101 ceph-mon[116267]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:19 smithi101 ceph-mon[116267]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:06:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:19 smithi101 ceph-mon[116267]: osdmap e17: 3 total, 2 up, 3 in 2023-06-22T18:06:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:19 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:19 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:19.402 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:19 smithi039 ceph-mon[109976]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:19.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:19 smithi039 ceph-mon[109976]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:06:19.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:19 smithi039 ceph-mon[109976]: osdmap e17: 3 total, 2 up, 3 in 2023-06-22T18:06:19.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:19.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:19.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:19 smithi133 ceph-mon[116867]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-06-22T18:06:19.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:19 smithi133 ceph-mon[116867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:06:19.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:19 smithi133 ceph-mon[116867]: osdmap e17: 3 total, 2 up, 3 in 2023-06-22T18:06:19.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:19 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:19.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:19 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:19.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2023-06-22T18:06:20.457 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:06:20.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:06:20.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:06:20.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416] boot 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: osdmap e18: 3 total, 3 up, 3 in 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:20.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:20.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:06:20.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:06:20.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416] boot 2023-06-22T18:06:20.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: osdmap e18: 3 total, 3 up, 3 in 2023-06-22T18:06:20.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:20.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:20.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:20.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:20.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:06:20.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:06:20.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: osd.2 [v2:172.21.15.133:6800/3165338416,v1:172.21.15.133:6801/3165338416] boot 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: osdmap e18: 3 total, 3 up, 3 in 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:20.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:20.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:20.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi133 2023-06-22T18:06:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi133 to 17172M 2023-06-22T18:06:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: pgmap v81: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-06-22T18:06:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-06-22T18:06:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: osdmap e19: 3 total, 3 up, 3 in 2023-06-22T18:06:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-22T18:06:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi133 2023-06-22T18:06:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi133 to 17172M 2023-06-22T18:06:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: pgmap v81: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-06-22T18:06:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-06-22T18:06:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: osdmap e19: 3 total, 3 up, 3 in 2023-06-22T18:06:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-22T18:06:21.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi133 2023-06-22T18:06:21.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi133 to 17172M 2023-06-22T18:06:21.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: pgmap v81: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:21.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2023-06-22T18:06:21.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2023-06-22T18:06:21.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: osdmap e19: 3 total, 3 up, 3 in 2023-06-22T18:06:21.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-22T18:06:21.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.877+0000 7fc34886c700 1 -- 172.21.15.39:0/527970741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fe7a0 msgr2=0x7fc344105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:21.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.877+0000 7fc34886c700 1 --2- 172.21.15.39:0/527970741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fe7a0 0x7fc344105060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009d60 tx=0x7fc33c004d10 comp rx=0 tx=0).stop 2023-06-22T18:06:21.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.877+0000 7fc34886c700 1 -- 172.21.15.39:0/527970741 shutdown_connections 2023-06-22T18:06:21.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.877+0000 7fc34886c700 1 --2- 172.21.15.39:0/527970741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fe7a0 0x7fc344105060 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:21.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.877+0000 7fc34886c700 1 --2- 172.21.15.39:0/527970741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3440fcc00 0x7fc3440fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:21.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.878+0000 7fc34886c700 1 --2- 172.21.15.39:0/527970741 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fde10 0x7fc3440fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:21.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.878+0000 7fc34886c700 1 -- 172.21.15.39:0/527970741 >> 172.21.15.39:0/527970741 conn(0x7fc3440f8320 msgr2=0x7fc3440fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:06:21.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.878+0000 7fc34886c700 1 -- 172.21.15.39:0/527970741 shutdown_connections 2023-06-22T18:06:21.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.878+0000 7fc34886c700 1 -- 172.21.15.39:0/527970741 wait complete. 2023-06-22T18:06:21.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.878+0000 7fc34886c700 1 Processor -- start 2023-06-22T18:06:21.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 -- start start 2023-06-22T18:06:21.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 0x7fc34410f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:21.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3440fde10 0x7fc34410fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:21.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fe7a0 0x7fc344116ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:21.885 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441174d0 con 0x7fc3440fcc00 2023-06-22T18:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344117640 con 0x7fc3440fe7a0 2023-06-22T18:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34886c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344117800 con 0x7fc3440fde10 2023-06-22T18:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 0x7fc34410f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 0x7fc34410f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52878/0 (socket says 172.21.15.39:52878) 2023-06-22T18:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc34259c700 1 -- 172.21.15.39:0/3331169978 learned_addr learned my addr 172.21.15.39:0/3331169978 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:06:21.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.879+0000 7fc341d9b700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3440fde10 0x7fc34410fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc342d9d700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fe7a0 0x7fc344116ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34259c700 1 -- 172.21.15.39:0/3331169978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3440fde10 msgr2=0x7fc34410fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34259c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3440fde10 0x7fc34410fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34259c700 1 -- 172.21.15.39:0/3331169978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fe7a0 msgr2=0x7fc344116ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34259c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fe7a0 0x7fc344116ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:21.887 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34259c700 1 -- 172.21.15.39:0/3331169978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338005040 con 0x7fc3440fcc00 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc342d9d700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fe7a0 0x7fc344116ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34259c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 0x7fc34410f8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc33400ebf0 tx=0x7fc33400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334007cc0 con 0x7fc3440fcc00 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c009940 con 0x7fc3440fcc00 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc33400f040 con 0x7fc3440fcc00 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33400c8f0 con 0x7fc3440fcc00 2023-06-22T18:06:21.888 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.880+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344117e60 con 0x7fc3440fcc00 2023-06-22T18:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.882+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344108400 con 0x7fc3440fcc00 2023-06-22T18:06:21.889 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.883+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 15) v1 ==== 129145+0+0 (secure 0 0 0) 0x7fc33400cd20 con 0x7fc3440fcc00 2023-06-22T18:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.885+0000 7fc332ffd700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc32c09b5f0 0x7fc32c09daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.885+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3233+0+0 (secure 0 0 0) 0x7fc3340c3460 con 0x7fc3440fcc00 2023-06-22T18:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.885+0000 7fc341d9b700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc32c09b5f0 0x7fc32c09daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.886+0000 7fc341d9b700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc32c09b5f0 0x7fc32c09daa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc338009fd0 tx=0x7fc33800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:06:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:21.889+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc33404da00 con 0x7fc3440fcc00 2023-06-22T18:06:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:22.121+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc344070c90 con 0x7fc32c09b5f0 2023-06-22T18:06:22.215 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:21 smithi039 sudo[129947]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-22T18:06:22.216 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:21 smithi039 sudo[129947]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:22.216 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:21 smithi039 sudo[129947]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:22.216 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:21 smithi039 sudo[129947]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:22.224 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:22.220+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fc3340904a0 con 0x7fc3440fcc00 2023-06-22T18:06:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-22T18:06:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:22 smithi133 ceph-mon[116867]: osdmap e20: 3 total, 3 up, 3 in 2023-06-22T18:06:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:22 smithi133 ceph-mon[116867]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:22.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:06:22.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:06:22.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:22.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-22T18:06:22.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:22 smithi039 ceph-mon[109976]: osdmap e20: 3 total, 3 up, 3 in 2023-06-22T18:06:22.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:22 smithi039 ceph-mon[109976]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:22.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:06:22.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:06:22.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:22.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-22T18:06:22.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:22 smithi101 ceph-mon[116267]: osdmap e20: 3 total, 3 up, 3 in 2023-06-22T18:06:22.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:22 smithi101 ceph-mon[116267]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:22.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:06:22.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:06:22.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:22.944 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:22 smithi039 sudo[130007]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-22T18:06:22.944 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:22 smithi039 sudo[130007]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:22.945 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:22 smithi039 sudo[130007]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:22.945 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:06:22 smithi039 sudo[130007]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:23.052 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:22 smithi101 sudo[125389]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-22T18:06:23.053 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:22 smithi101 sudo[125389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:23.053 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:22 smithi101 sudo[125389]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:23.053 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:22 smithi101 sudo[125389]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:23.353 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:23 smithi101 sudo[125392]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-22T18:06:23.353 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:23 smithi101 sudo[125392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:23.353 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:23 smithi101 sudo[125392]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:23.353 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:06:23 smithi101 sudo[125392]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:23.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:23 smithi101 ceph-mon[116267]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:06:23.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:23 smithi101 ceph-mon[116267]: osdmap e21: 3 total, 3 up, 3 in 2023-06-22T18:06:23.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:23 smithi101 ceph-mon[116267]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-22T18:06:23.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:23 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:23.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:23 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:23.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127186]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-22T18:06:23.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:23.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127186]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:23.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127186]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:23.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:23 smithi133 ceph-mon[116867]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:06:23.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:23 smithi133 ceph-mon[116867]: osdmap e21: 3 total, 3 up, 3 in 2023-06-22T18:06:23.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:23 smithi133 ceph-mon[116867]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-22T18:06:23.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:23 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:23.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:23 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:23 smithi039 ceph-mon[109976]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:06:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:23 smithi039 ceph-mon[109976]: osdmap e21: 3 total, 3 up, 3 in 2023-06-22T18:06:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:23 smithi039 ceph-mon[109976]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-22T18:06:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:23 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:23.577 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:23 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:23.984 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127189]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-22T18:06:23.984 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127189]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:23.984 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127189]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:23.985 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:06:23 smithi133 sudo[127189]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:24.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130276]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-22T18:06:24.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:24.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130276]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:24.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130276]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:24 smithi101 ceph-mon[116267]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130313]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-22T18:06:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130313]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 sudo[130313]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:24 smithi039 ceph-mon[109976]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:25.211 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:24 smithi133 sudo[127192]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-22T18:06:25.212 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:24 smithi133 sudo[127192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:25.212 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:24 smithi133 sudo[127192]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:25.212 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:24 smithi133 sudo[127192]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:25.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 sudo[127195]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-22T18:06:25.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 sudo[127195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:25.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 sudo[127195]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:25.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 sudo[127195]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:25.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:25.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:25.707 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125395]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-22T18:06:25.707 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:25.707 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125395]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:25.708 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125395]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:25.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:25.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:25.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:25.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:25.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3879337515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e"}]: dispatch 2023-06-22T18:06:25.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:25.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:25.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:25.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:25.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:25.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3879337515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e"}]': finished 2023-06-22T18:06:25.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: osdmap e22: 4 total, 3 up, 4 in 2023-06-22T18:06:25.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:25 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:26.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125398]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-22T18:06:26.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-22T18:06:26.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125398]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-22T18:06:26.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 sudo[125398]: pam_unix(sudo:session): session closed for user root 2023-06-22T18:06:26.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:26.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:26.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:26.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3879337515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e"}]: dispatch 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:26.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3879337515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e"}]': finished 2023-06-22T18:06:26.106 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: osdmap e22: 4 total, 3 up, 4 in 2023-06-22T18:06:26.106 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:25 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:26.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:26.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:26.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:26.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:26.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:26.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:26.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:26.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:26.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:26.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:26.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3879337515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e"}]: dispatch 2023-06-22T18:06:26.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-22T18:06:26.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-22T18:06:26.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-22T18:06:26.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-22T18:06:26.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-22T18:06:26.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3879337515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e"}]': finished 2023-06-22T18:06:26.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: osdmap e22: 4 total, 3 up, 4 in 2023-06-22T18:06:26.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:25 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:26.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:26 smithi133 ceph-mon[116867]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:26.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:26 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1843109447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:26 smithi101 ceph-mon[116267]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:26 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1843109447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:27.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:26 smithi039 ceph-mon[109976]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:27.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:26 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1843109447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:29.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:28 smithi101 ceph-mon[116267]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:29.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:28 smithi039 ceph-mon[109976]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:29.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:28 smithi133 ceph-mon[116867]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:31.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:30 smithi101 ceph-mon[116267]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:31.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:30 smithi039 ceph-mon[109976]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:31.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:30 smithi133 ceph-mon[116867]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:32 smithi039 ceph-mon[109976]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-06-22T18:06:33.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:32 smithi101 ceph-mon[116267]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-06-22T18:06:33.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:32 smithi133 ceph-mon[116867]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-06-22T18:06:34.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:33 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-22T18:06:34.093 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:33 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:34.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:33 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-22T18:06:34.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:33 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:34.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:33 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-22T18:06:34.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:33 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:35.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:34 smithi101 ceph-mon[116267]: Deploying daemon osd.3 on smithi039 2023-06-22T18:06:35.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:34 smithi101 ceph-mon[116267]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-06-22T18:06:35.163 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:34 smithi039 ceph-mon[109976]: Deploying daemon osd.3 on smithi039 2023-06-22T18:06:35.163 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:34 smithi039 ceph-mon[109976]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-06-22T18:06:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:34 smithi133 ceph-mon[116867]: Deploying daemon osd.3 on smithi039 2023-06-22T18:06:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:34 smithi133 ceph-mon[116867]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-06-22T18:06:36.824 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:36 smithi039 ceph-mon[109976]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-06-22T18:06:37.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:36 smithi101 ceph-mon[116267]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-06-22T18:06:37.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:36 smithi133 ceph-mon[116867]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-06-22T18:06:39.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:38 smithi101 ceph-mon[116267]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 773 B/s rd, 22 KiB/s wr, 2 op/s 2023-06-22T18:06:39.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:38 smithi039 ceph-mon[109976]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 773 B/s rd, 22 KiB/s wr, 2 op/s 2023-06-22T18:06:39.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:38 smithi133 ceph-mon[116867]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 773 B/s rd, 22 KiB/s wr, 2 op/s 2023-06-22T18:06:40.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:40.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:40.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:40.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:40 smithi101 ceph-mon[116267]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-22T18:06:40.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:40.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:40.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:40.855 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:40 smithi039 ceph-mon[109976]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-22T18:06:40.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:40.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:40.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:40.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:40 smithi133 ceph-mon[116867]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-22T18:06:42.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:42 smithi039 ceph-mon[109976]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-22T18:06:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:42.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:42.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:42 smithi133 ceph-mon[116867]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-22T18:06:42.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:42.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:42.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:42.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:42.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.981+0000 7fc332ffd700 1 -- 172.21.15.39:0/3331169978 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc344070c90 con 0x7fc32c09b5f0 2023-06-22T18:06:42.986 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2023-06-22T18:06:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc32c09b5f0 msgr2=0x7fc32c09daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc32c09b5f0 0x7fc32c09daa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc338009fd0 tx=0x7fc33800b040 comp rx=0 tx=0).stop 2023-06-22T18:06:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 msgr2=0x7fc34410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 0x7fc34410f8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc33400ebf0 tx=0x7fc33400b040 comp rx=0 tx=0).stop 2023-06-22T18:06:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 shutdown_connections 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc32c09b5f0 0x7fc32c09daa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3440fe7a0 0x7fc344116ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3440fcc00 0x7fc34410f8f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 --2- 172.21.15.39:0/3331169978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3440fde10 0x7fc34410fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 >> 172.21.15.39:0/3331169978 conn(0x7fc3440f8320 msgr2=0x7fc3440fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 shutdown_connections 2023-06-22T18:06:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:06:42.984+0000 7fc34886c700 1 -- 172.21.15.39:0/3331169978 wait complete. 2023-06-22T18:06:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:42 smithi101 ceph-mon[116267]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-22T18:06:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:43.559 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.3.service 2023-06-22T18:06:43.563 INFO:tasks.cephadm:Deploying osd.4 on smithi101 with /dev/vg_nvme/lv_3... 2023-06-22T18:06:43.564 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-22T18:06:43.651 INFO:journalctl@ceph.osd.3.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:06:44.161 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:06:44.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:43 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:44.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:43 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:44.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:43 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:44.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:43 smithi133 ceph-mon[116867]: from='osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-22T18:06:44.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:43 smithi133 ceph-mon[116867]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-22T18:06:44.239 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:43 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:44.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:43 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:44.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:43 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:44.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:43 smithi039 ceph-mon[109976]: from='osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-22T18:06:44.240 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:43 smithi039 ceph-mon[109976]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-22T18:06:44.338 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:44.338 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:44.338 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:06:44.338 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:43 smithi101 ceph-mon[116267]: from='osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-22T18:06:44.338 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:43 smithi101 ceph-mon[116267]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-22T18:06:45.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:44 smithi101 ceph-mon[116267]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:45.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:44 smithi101 ceph-mon[116267]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-22T18:06:45.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:44 smithi101 ceph-mon[116267]: osdmap e23: 4 total, 3 up, 4 in 2023-06-22T18:06:45.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:44 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:45.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:44 smithi101 ceph-mon[116267]: from='osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:06:45.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:44 smithi101 ceph-mon[116267]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:06:45.190 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:06:44 smithi039 conmon[133234]: 2023-06-22T18:06:44.991+0000 7f4a3c28e700 -1 osd.3 0 waiting for initial osdmap 2023-06-22T18:06:45.190 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:06:45 smithi039 conmon[133234]: 2023-06-22T18:06:45.001+0000 7f4a36087700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:06:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:44 smithi039 ceph-mon[109976]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:44 smithi039 ceph-mon[109976]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-22T18:06:45.190 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:44 smithi039 ceph-mon[109976]: osdmap e23: 4 total, 3 up, 4 in 2023-06-22T18:06:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:44 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:44 smithi039 ceph-mon[109976]: from='osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:06:45.191 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:44 smithi039 ceph-mon[109976]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:06:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:44 smithi133 ceph-mon[116867]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:44 smithi133 ceph-mon[116867]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-22T18:06:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:44 smithi133 ceph-mon[116867]: osdmap e23: 4 total, 3 up, 4 in 2023-06-22T18:06:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:44 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:44 smithi133 ceph-mon[116867]: from='osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:06:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:44 smithi133 ceph-mon[116867]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:06:46.079 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-22T18:06:46.079 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-22T18:06:46.079 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-06-22T18:06:46.079 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-06-22T18:06:46.080 INFO:teuthology.orchestra.run.smithi101.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.010947 s, 958 MB/s 2023-06-22T18:06:46.080 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 2023-06-22T18:06:46.080 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-06-22T18:06:46.264 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:45 smithi039 ceph-mon[109976]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:06:46.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:45 smithi039 ceph-mon[109976]: osdmap e24: 4 total, 3 up, 4 in 2023-06-22T18:06:46.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:46.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:46.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:45 smithi039 ceph-mon[109976]: osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438] boot 2023-06-22T18:06:46.305 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:45 smithi101 ceph-mon[116267]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:06:46.305 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:45 smithi101 ceph-mon[116267]: osdmap e24: 4 total, 3 up, 4 in 2023-06-22T18:06:46.306 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:46.306 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:46.306 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:45 smithi101 ceph-mon[116267]: osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438] boot 2023-06-22T18:06:46.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:45 smithi133 ceph-mon[116867]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:06:46.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:45 smithi133 ceph-mon[116867]: osdmap e24: 4 total, 3 up, 4 in 2023-06-22T18:06:46.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:46.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:46.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:45 smithi133 ceph-mon[116867]: osd.3 [v2:172.21.15.39:6810/3695104438,v1:172.21.15.39:6811/3695104438] boot 2023-06-22T18:06:46.893 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_3 2023-06-22T18:06:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:06:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:06:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: osdmap e25: 4 total, 4 up, 4 in 2023-06-22T18:06:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:47.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:47 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:06:47.501 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:06:47.502 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:47.502 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: osdmap e25: 4 total, 4 up, 4 in 2023-06-22T18:06:47.502 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:47.502 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.502 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.502 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:47.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:47.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:47.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:47.503 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.504 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:06:47.504 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:06:47.504 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-06-22T18:06:47.504 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: osdmap e25: 4 total, 4 up, 4 in 2023-06-22T18:06:47.504 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:47.505 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:06:47.506 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:06:47.506 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:06:48.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:48 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi039 2023-06-22T18:06:48.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:48 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi039 to 8633M 2023-06-22T18:06:48.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:48 smithi101 ceph-mon[116267]: osdmap e26: 4 total, 4 up, 4 in 2023-06-22T18:06:48.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:48 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi039 2023-06-22T18:06:48.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:48 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi039 to 8633M 2023-06-22T18:06:48.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:48 smithi039 ceph-mon[109976]: osdmap e26: 4 total, 4 up, 4 in 2023-06-22T18:06:48.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:48 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi039 2023-06-22T18:06:48.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:48 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi039 to 8633M 2023-06-22T18:06:48.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:48 smithi133 ceph-mon[116867]: osdmap e26: 4 total, 4 up, 4 in 2023-06-22T18:06:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 -- 172.21.15.101:0/1548354283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fde40 msgr2=0x7f33d40fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 --2- 172.21.15.101:0/1548354283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fde40 0x7f33d40fe290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f33c80049c0 tx=0x7f33c8009e30 comp rx=0 tx=0).stop 2023-06-22T18:06:48.957 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 -- 172.21.15.101:0/1548354283 shutdown_connections 2023-06-22T18:06:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 --2- 172.21.15.101:0/1548354283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d40fe7d0 0x7f33d4105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 --2- 172.21.15.101:0/1548354283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d40fcc30 0x7f33d40fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 --2- 172.21.15.101:0/1548354283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fde40 0x7f33d40fe290 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 -- 172.21.15.101:0/1548354283 >> 172.21.15.101:0/1548354283 conn(0x7f33d40f8350 msgr2=0x7f33d40fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:06:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 -- 172.21.15.101:0/1548354283 shutdown_connections 2023-06-22T18:06:48.958 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.954+0000 7f33d90df700 1 -- 172.21.15.101:0/1548354283 wait complete. 2023-06-22T18:06:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 Processor -- start 2023-06-22T18:06:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 -- start start 2023-06-22T18:06:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 0x7f33d410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d40fde40 0x7f33d410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d40fe7d0 0x7f33d4114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:48.960 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d4115580 con 0x7f33d40fe7d0 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d41156f0 con 0x7f33d40fcc30 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d90df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d41158b0 con 0x7f33d40fde40 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 0x7f33d410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 0x7f33d410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44100/0 (socket says 172.21.15.101:44100) 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 -- 172.21.15.101:0/2791299576 learned_addr learned my addr 172.21.15.101:0/2791299576 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d259c700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d40fde40 0x7f33d410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:48.961 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 -- 172.21.15.101:0/2791299576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d40fde40 msgr2=0x7f33d410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d40fde40 0x7f33d410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 -- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d40fe7d0 msgr2=0x7f33d4114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d40fe7d0 0x7f33d4114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 -- 172.21.15.101:0/2791299576 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c8005040 con 0x7f33d40fcc30 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.955+0000 7f33d2d9d700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 0x7f33d410d9a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33c400b810 tx=0x7f33c4009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.956+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c400f510 con 0x7f33d40fcc30 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.956+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d4115b90 con 0x7f33d40fcc30 2023-06-22T18:06:48.962 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.956+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33c400fb40 con 0x7f33d40fcc30 2023-06-22T18:06:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.956+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4011c30 con 0x7f33d40fcc30 2023-06-22T18:06:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.956+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d4116080 con 0x7f33d40fcc30 2023-06-22T18:06:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.956+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d41084c0 con 0x7f33d40fcc30 2023-06-22T18:06:48.963 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.957+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f33c400fcb0 con 0x7f33d40fcc30 2023-06-22T18:06:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.958+0000 7f33bb7fe700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f33bc09b680 0x7f33bc09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:06:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.958+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3694+0+0 (secure 0 0 0) 0x7f33c40c4600 con 0x7f33d40fcc30 2023-06-22T18:06:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.959+0000 7f33d259c700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f33bc09b680 0x7f33bc09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:06:48.964 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.959+0000 7f33d259c700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f33bc09b680 0x7f33bc09db30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33c8004fb0 tx=0x7f33c8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:06:48.965 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:48.962+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f33c4091070 con 0x7f33d40fcc30 2023-06-22T18:06:49.159 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:06:49.156+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f33d405de70 con 0x7f33bc09b680 2023-06-22T18:06:49.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:49 smithi101 ceph-mon[116267]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:06:49.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:49 smithi101 ceph-mon[116267]: osdmap e27: 4 total, 4 up, 4 in 2023-06-22T18:06:49.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:49 smithi039 ceph-mon[109976]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:06:49.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:49 smithi039 ceph-mon[109976]: osdmap e27: 4 total, 4 up, 4 in 2023-06-22T18:06:49.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:49 smithi133 ceph-mon[116867]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:06:49.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:49 smithi133 ceph-mon[116867]: osdmap e27: 4 total, 4 up, 4 in 2023-06-22T18:06:50.195 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:50 smithi101 ceph-mon[116267]: osdmap e28: 4 total, 4 up, 4 in 2023-06-22T18:06:50.196 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:06:50.196 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:06:50.196 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:50.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:50 smithi039 ceph-mon[109976]: osdmap e28: 4 total, 4 up, 4 in 2023-06-22T18:06:50.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:06:50.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:06:50.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:50.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:50 smithi133 ceph-mon[116867]: osdmap e28: 4 total, 4 up, 4 in 2023-06-22T18:06:50.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:06:50.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:06:50.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:06:51.182 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:51 smithi101 ceph-mon[116267]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:06:51.183 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:51 smithi101 ceph-mon[116267]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:06:51.183 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:51 smithi101 ceph-mon[116267]: osdmap e29: 4 total, 4 up, 4 in 2023-06-22T18:06:51.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:51 smithi039 ceph-mon[109976]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:06:51.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:51 smithi039 ceph-mon[109976]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:06:51.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:51 smithi039 ceph-mon[109976]: osdmap e29: 4 total, 4 up, 4 in 2023-06-22T18:06:51.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:51 smithi133 ceph-mon[116867]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:06:51.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:51 smithi133 ceph-mon[116867]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:06:51.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:51 smithi133 ceph-mon[116867]: osdmap e29: 4 total, 4 up, 4 in 2023-06-22T18:06:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:52 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/1292628902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]: dispatch 2023-06-22T18:06:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:52 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]: dispatch 2023-06-22T18:06:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:52 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]': finished 2023-06-22T18:06:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:52 smithi101 ceph-mon[116267]: osdmap e30: 5 total, 4 up, 5 in 2023-06-22T18:06:52.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:06:52.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:52 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/1292628902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]: dispatch 2023-06-22T18:06:52.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:52 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]: dispatch 2023-06-22T18:06:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:52 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]': finished 2023-06-22T18:06:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:52 smithi039 ceph-mon[109976]: osdmap e30: 5 total, 4 up, 5 in 2023-06-22T18:06:52.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:06:52.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:52 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/1292628902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]: dispatch 2023-06-22T18:06:52.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:52 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]: dispatch 2023-06-22T18:06:52.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:52 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9ecace5-1032-4f25-a5f3-84ec819c83ee"}]': finished 2023-06-22T18:06:52.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:52 smithi133 ceph-mon[116867]: osdmap e30: 5 total, 4 up, 5 in 2023-06-22T18:06:52.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:06:53.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:53 smithi101 ceph-mon[116267]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2023-06-22T18:06:53.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:53 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/835086353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:53.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:53 smithi039 ceph-mon[109976]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2023-06-22T18:06:53.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:53 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/835086353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:53.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:53 smithi133 ceph-mon[116867]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2023-06-22T18:06:53.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:53 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/835086353' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:06:55.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:55 smithi101 ceph-mon[116267]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-06-22T18:06:55.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:55 smithi039 ceph-mon[109976]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-06-22T18:06:55.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:55 smithi133 ceph-mon[116867]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-06-22T18:06:57.320 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:57 smithi101 ceph-mon[116267]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-06-22T18:06:57.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:57 smithi039 ceph-mon[109976]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-06-22T18:06:57.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:57 smithi133 ceph-mon[116867]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-06-22T18:06:59.357 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:06:59 smithi101 ceph-mon[116267]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-22T18:06:59.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:06:59 smithi039 ceph-mon[109976]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-22T18:06:59.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:06:59 smithi133 ceph-mon[116867]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-22T18:07:00.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-22T18:07:00.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:00.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-22T18:07:00.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:00.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-22T18:07:00.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:01.311 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:01 smithi101 ceph-mon[116267]: Deploying daemon osd.4 on smithi101 2023-06-22T18:07:01.311 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:01 smithi101 ceph-mon[116267]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:01.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:01 smithi039 ceph-mon[109976]: Deploying daemon osd.4 on smithi101 2023-06-22T18:07:01.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:01 smithi039 ceph-mon[109976]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:01.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:01 smithi133 ceph-mon[116867]: Deploying daemon osd.4 on smithi101 2023-06-22T18:07:01.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:01 smithi133 ceph-mon[116867]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:02.147 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:02 smithi101 ceph-mon[116267]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:02.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:02 smithi039 ceph-mon[109976]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:02.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:02 smithi133 ceph-mon[116867]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:04 smithi133 ceph-mon[116867]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:04 smithi101 ceph-mon[116267]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:04 smithi039 ceph-mon[109976]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:05.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:07.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:06 smithi101 ceph-mon[116267]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:07.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:06 smithi039 ceph-mon[109976]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:07.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:06 smithi133 ceph-mon[116867]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:08.033 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 4 on host 'smithi101' 2023-06-22T18:07:08.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.030+0000 7f33bb7fe700 1 -- 172.21.15.101:0/2791299576 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f33d405de70 con 0x7f33bc09b680 2023-06-22T18:07:08.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f33bc09b680 msgr2=0x7f33bc09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:08.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f33bc09b680 0x7f33bc09db30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33c8004fb0 tx=0x7f33c8009e30 comp rx=0 tx=0).stop 2023-06-22T18:07:08.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 msgr2=0x7f33d410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:08.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 0x7f33d410d9a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33c400b810 tx=0x7f33c4009220 comp rx=0 tx=0).stop 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 shutdown_connections 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f33bc09b680 0x7f33bc09db30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d40fe7d0 0x7f33d4114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33d40fcc30 0x7f33d410d9a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 --2- 172.21.15.101:0/2791299576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33d40fde40 0x7f33d410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 >> 172.21.15.101:0/2791299576 conn(0x7f33d40f8350 msgr2=0x7f33d40fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:07:08.036 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 shutdown_connections 2023-06-22T18:07:08.037 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:07:08.032+0000 7f33d90df700 1 -- 172.21.15.101:0/2791299576 wait complete. 2023-06-22T18:07:08.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:08.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:08.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:08.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:08.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:08.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:08.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:08.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:08.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:08.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:08.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:08.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:08.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:08.629 DEBUG:teuthology.orchestra.run.smithi101:osd.4> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.4.service 2023-06-22T18:07:08.631 INFO:tasks.cephadm:Deploying osd.5 on smithi133 with /dev/vg_nvme/lv_3... 2023-06-22T18:07:08.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-22T18:07:08.660 INFO:journalctl@ceph.osd.4.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:07:09.229 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:07:09.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:09 smithi101 ceph-mon[116267]: from='osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-22T18:07:09.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:09 smithi101 ceph-mon[116267]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-22T18:07:09.407 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:09 smithi133 ceph-mon[116867]: from='osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-22T18:07:09.408 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:09 smithi133 ceph-mon[116867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-22T18:07:09.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:09 smithi039 ceph-mon[109976]: from='osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-22T18:07:09.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:09 smithi039 ceph-mon[109976]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-22T18:07:10.232 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: osdmap e31: 5 total, 4 up, 5 in 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: from='osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:07:10.233 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: osdmap e32: 5 total, 4 up, 5 in 2023-06-22T18:07:10.234 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:10 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:10.234 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:07:10 smithi101 conmon[129640]: 2023-06-22T18:07:10.132+0000 7f47e6d09700 -1 osd.4 0 waiting for initial osdmap 2023-06-22T18:07:10.234 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:07:10 smithi101 conmon[129640]: 2023-06-22T18:07:10.139+0000 7f47e0b02700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:07:10.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-22T18:07:10.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: osdmap e31: 5 total, 4 up, 5 in 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: from='osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: osdmap e32: 5 total, 4 up, 5 in 2023-06-22T18:07:10.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:10.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-22T18:07:10.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: osdmap e31: 5 total, 4 up, 5 in 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: from='osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: osdmap e32: 5 total, 4 up, 5 in 2023-06-22T18:07:10.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:10 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.047 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-22T18:07:11.047 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-22T18:07:11.048 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-06-22T18:07:11.048 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-06-22T18:07:11.048 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109585 s, 957 MB/s 2023-06-22T18:07:11.048 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-06-22T18:07:11.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.448 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.448 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:11.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2023-06-22T18:07:12.235 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:07:12.415 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:07:12.415 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:07:12.415 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315] boot 2023-06-22T18:07:12.416 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: osdmap e33: 5 total, 5 up, 5 in 2023-06-22T18:07:12.416 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:12.416 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi101 2023-06-22T18:07:12.416 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.416 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.416 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi101 to 8586M 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:12 smithi133 ceph-mon[116867]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:12.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:07:12.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315] boot 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: osdmap e33: 5 total, 5 up, 5 in 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi101 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi101 to 8586M 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:12 smithi039 ceph-mon[109976]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:12.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:07:12.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:07:12.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: osd.4 [v2:172.21.15.101:6808/2726745315,v1:172.21.15.101:6809/2726745315] boot 2023-06-22T18:07:12.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: osdmap e33: 5 total, 5 up, 5 in 2023-06-22T18:07:12.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-22T18:07:12.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi101 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi101 to 8586M 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:12.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:12.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:12.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:12 smithi101 ceph-mon[116267]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-06-22T18:07:13.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:13 smithi039 ceph-mon[109976]: osdmap e34: 5 total, 5 up, 5 in 2023-06-22T18:07:13.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:13 smithi133 ceph-mon[116867]: osdmap e34: 5 total, 5 up, 5 in 2023-06-22T18:07:13.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:13 smithi101 ceph-mon[116267]: osdmap e34: 5 total, 5 up, 5 in 2023-06-22T18:07:13.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 -- 172.21.15.133:0/3479811869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608100060 msgr2=0x7fc608102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/3479811869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608100060 0x7fc608102440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0004900 tx=0x7fc5f0009e30 comp rx=0 tx=0).stop 2023-06-22T18:07:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 -- 172.21.15.133:0/3479811869 shutdown_connections 2023-06-22T18:07:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/3479811869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc608102980 0x7fc608104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/3479811869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc6080fd740 0x7fc6080ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/3479811869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608100060 0x7fc608102440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 -- 172.21.15.133:0/3479811869 >> 172.21.15.133:0/3479811869 conn(0x7fc6080f7220 msgr2=0x7fc6080f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 -- 172.21.15.133:0/3479811869 shutdown_connections 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 -- 172.21.15.133:0/3479811869 wait complete. 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.705+0000 7fc60d4d2700 1 Processor -- start 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 -- start start 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080fd740 0x7fc6080fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc608100060 0x7fc6080fb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:13.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 0x7fc6080fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080fc300 con 0x7fc6080fd740 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080fc470 con 0x7fc608100060 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc608117960 con 0x7fc608102980 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 0x7fc6080fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 0x7fc6080fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47574/0 (socket says 172.21.15.133:47574) 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 -- 172.21.15.133:0/2024293503 learned_addr learned my addr 172.21.15.133:0/2024293503 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc5fffff700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc608100060 0x7fc6080fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:13.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 -- 172.21.15.133:0/2024293503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc608100060 msgr2=0x7fc6080fb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc608100060 0x7fc6080fb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 -- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080fd740 msgr2=0x7fc6080fcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080fd740 0x7fc6080fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 -- 172.21.15.133:0/2024293503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8005040 con 0x7fc608102980 2023-06-22T18:07:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc6077fe700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 0x7fc6080fbb90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc600001d80 tx=0x7fc600009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:07:13.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600007b30 con 0x7fc608102980 2023-06-22T18:07:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc60000ee30 con 0x7fc608102980 2023-06-22T18:07:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f0005040 con 0x7fc608102980 2023-06-22T18:07:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000097c0 con 0x7fc608102980 2023-06-22T18:07:13.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.706+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc608118040 con 0x7fc608102980 2023-06-22T18:07:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.707+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6080f8d50 con 0x7fc608102980 2023-06-22T18:07:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.708+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fc60000e960 con 0x7fc608102980 2023-06-22T18:07:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.709+0000 7fc5ff7fe700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc5f409b780 0x7fc5f409dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.709+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4097+0+0 (secure 0 0 0) 0x7fc6000c4380 con 0x7fc608102980 2023-06-22T18:07:13.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.709+0000 7fc606ffd700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc5f409b780 0x7fc5f409dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.710+0000 7fc606ffd700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc5f409b780 0x7fc5f409dc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009fd0 tx=0x7fc5f8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:07:13.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.713+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc600090bb0 con 0x7fc608102980 2023-06-22T18:07:13.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:13.901+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc60802b410 con 0x7fc5f409b780 2023-06-22T18:07:14.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:14 smithi039 ceph-mon[109976]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:14.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:14 smithi039 ceph-mon[109976]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:07:14.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:07:14.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:07:14.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:14.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:14 smithi133 ceph-mon[116867]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:14.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:14 smithi133 ceph-mon[116867]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:07:14.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:14 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:07:14.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:14 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:07:14.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:14 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:14.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:14 smithi101 ceph-mon[116267]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:14.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:14 smithi101 ceph-mon[116267]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:07:14.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:14 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:07:14.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:14 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:07:14.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:14 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:17.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:16 smithi101 ceph-mon[116267]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:16 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/253927244' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]: dispatch 2023-06-22T18:07:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:16 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]: dispatch 2023-06-22T18:07:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:16 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]': finished 2023-06-22T18:07:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:16 smithi101 ceph-mon[116267]: osdmap e35: 6 total, 5 up, 6 in 2023-06-22T18:07:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:17.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:16 smithi039 ceph-mon[109976]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:17.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:16 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/253927244' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]: dispatch 2023-06-22T18:07:17.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:16 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]: dispatch 2023-06-22T18:07:17.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:16 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]': finished 2023-06-22T18:07:17.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:16 smithi039 ceph-mon[109976]: osdmap e35: 6 total, 5 up, 6 in 2023-06-22T18:07:17.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:16 smithi133 ceph-mon[116867]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:16 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/253927244' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]: dispatch 2023-06-22T18:07:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:16 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]: dispatch 2023-06-22T18:07:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:16 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "379ed499-68d9-4032-bafd-31cb02995388"}]': finished 2023-06-22T18:07:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:16 smithi133 ceph-mon[116867]: osdmap e35: 6 total, 5 up, 6 in 2023-06-22T18:07:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:18.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:17 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/1708025359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:07:18.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:17 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/1708025359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:07:18.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:17 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/1708025359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:07:19.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:18 smithi101 ceph-mon[116267]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:19.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:18 smithi039 ceph-mon[109976]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:19.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:18 smithi133 ceph-mon[116867]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:21.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:20 smithi101 ceph-mon[116267]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:21.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:20 smithi039 ceph-mon[109976]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:21.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:20 smithi133 ceph-mon[116867]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:23.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:22 smithi133 ceph-mon[116867]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:23.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:22 smithi101 ceph-mon[116267]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:22 smithi039 ceph-mon[109976]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:25.087 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:24 smithi133 ceph-mon[116867]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:25.087 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-22T18:07:25.088 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:24 smithi101 ceph-mon[116267]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-22T18:07:25.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:25.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:24 smithi039 ceph-mon[109976]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-22T18:07:25.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:26.021 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:25 smithi133 ceph-mon[116867]: Deploying daemon osd.5 on smithi133 2023-06-22T18:07:26.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:25 smithi101 ceph-mon[116267]: Deploying daemon osd.5 on smithi133 2023-06-22T18:07:26.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:25 smithi039 ceph-mon[109976]: Deploying daemon osd.5 on smithi133 2023-06-22T18:07:27.036 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:26 smithi133 ceph-mon[116867]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:27.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:26 smithi101 ceph-mon[116267]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:27.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:26 smithi039 ceph-mon[109976]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:29.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:28 smithi101 ceph-mon[116267]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:29.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:28 smithi039 ceph-mon[109976]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:29.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:28 smithi133 ceph-mon[116867]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:30.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:30 smithi133 ceph-mon[116867]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:30.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:30 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:30.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:30 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:30.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:30 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:31.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:30 smithi101 ceph-mon[116267]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:31.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:30 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:31.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:30 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:31.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:30 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:31.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:30 smithi039 ceph-mon[109976]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:31.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:31.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:31.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:30 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:32 smithi133 ceph-mon[116867]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:32 smithi101 ceph-mon[116267]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:33.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:33.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:33.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:32 smithi039 ceph-mon[109976]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:33.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:33.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:33.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:33.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.274+0000 7fc5ff7fe700 1 -- 172.21.15.133:0/2024293503 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc60802b410 con 0x7fc5f409b780 2023-06-22T18:07:33.278 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 5 on host 'smithi133' 2023-06-22T18:07:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc5f409b780 msgr2=0x7fc5f409dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc5f409b780 0x7fc5f409dc30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009fd0 tx=0x7fc5f8009e30 comp rx=0 tx=0).stop 2023-06-22T18:07:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 msgr2=0x7fc6080fbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 0x7fc6080fbb90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc600001d80 tx=0x7fc600009ee0 comp rx=0 tx=0).stop 2023-06-22T18:07:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 shutdown_connections 2023-06-22T18:07:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc5f409b780 0x7fc5f409dc30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc608102980 0x7fc6080fbb90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080fd740 0x7fc6080fcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 --2- 172.21.15.133:0/2024293503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc608100060 0x7fc6080fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 >> 172.21.15.133:0/2024293503 conn(0x7fc6080f7220 msgr2=0x7fc6080fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:07:33.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 shutdown_connections 2023-06-22T18:07:33.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:07:33.276+0000 7fc60d4d2700 1 -- 172.21.15.133:0/2024293503 wait complete. 2023-06-22T18:07:33.985 DEBUG:teuthology.orchestra.run.smithi133:osd.5> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.5.service 2023-06-22T18:07:33.987 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2023-06-22T18:07:33.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-22T18:07:34.017 INFO:journalctl@ceph.osd.5.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:07:34.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:34 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:34.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:34 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:34.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:34 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:34.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:34 smithi039 ceph-mon[109976]: from='osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-22T18:07:34.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:34 smithi039 ceph-mon[109976]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-22T18:07:34.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:34 smithi039 ceph-mon[109976]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:34.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:34 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:34.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:34 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:34.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:34 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:34.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:34 smithi133 ceph-mon[116867]: from='osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-22T18:07:34.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:34 smithi133 ceph-mon[116867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-22T18:07:34.570 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:34 smithi133 ceph-mon[116867]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:34.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:34 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:34 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:34 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:34 smithi101 ceph-mon[116267]: from='osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-22T18:07:34.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:34 smithi101 ceph-mon[116267]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-22T18:07:34.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:34 smithi101 ceph-mon[116267]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:34.767 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:07:35.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:35 smithi039 ceph-mon[109976]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-22T18:07:35.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:35 smithi039 ceph-mon[109976]: osdmap e36: 6 total, 5 up, 6 in 2023-06-22T18:07:35.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:35.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:35 smithi039 ceph-mon[109976]: from='osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:07:35.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:35 smithi039 ceph-mon[109976]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:07:35.519 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:35 smithi133 ceph-mon[116867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-22T18:07:35.520 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:35 smithi133 ceph-mon[116867]: osdmap e36: 6 total, 5 up, 6 in 2023-06-22T18:07:35.520 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:35.520 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:35 smithi133 ceph-mon[116867]: from='osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:07:35.521 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:35 smithi133 ceph-mon[116867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:07:35.521 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:07:35 smithi133 conmon[131441]: 2023-06-22T18:07:35.263+0000 7f2bec140700 -1 osd.5 0 waiting for initial osdmap 2023-06-22T18:07:35.521 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:07:35 smithi133 conmon[131441]: 2023-06-22T18:07:35.271+0000 7f2be8f3f700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:07:35.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:35 smithi101 ceph-mon[116267]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-22T18:07:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:35 smithi101 ceph-mon[116267]: osdmap e36: 6 total, 5 up, 6 in 2023-06-22T18:07:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:35 smithi101 ceph-mon[116267]: from='osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:07:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:35 smithi101 ceph-mon[116267]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:07:36.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:36 smithi133 ceph-mon[116867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:07:36.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:36 smithi133 ceph-mon[116867]: osdmap e37: 6 total, 5 up, 6 in 2023-06-22T18:07:36.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:36 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:36.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:36 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:36.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:36 smithi133 ceph-mon[116867]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:36.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:36 smithi101 ceph-mon[116267]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:07:36.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:36 smithi101 ceph-mon[116267]: osdmap e37: 6 total, 5 up, 6 in 2023-06-22T18:07:36.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:36 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:36.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:36 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:36.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:36 smithi101 ceph-mon[116267]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:36.659 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-22T18:07:36.660 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-22T18:07:36.660 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-06-22T18:07:36.660 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-06-22T18:07:36.660 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0405253 s, 259 MB/s 2023-06-22T18:07:36.660 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-06-22T18:07:36.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:36 smithi039 ceph-mon[109976]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:07:36.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:36 smithi039 ceph-mon[109976]: osdmap e37: 6 total, 5 up, 6 in 2023-06-22T18:07:36.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:36 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:36.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:36 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:36.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:36 smithi039 ceph-mon[109976]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-06-22T18:07:37.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2023-06-22T18:07:37.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:07:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:07:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441] boot 2023-06-22T18:07:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: osdmap e38: 6 total, 6 up, 6 in 2023-06-22T18:07:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi133 2023-06-22T18:07:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:37.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:37.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi133 to 8586M 2023-06-22T18:07:37.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.448 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:37.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:37.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:07:37.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:07:37.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:37.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441] boot 2023-06-22T18:07:37.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: osdmap e38: 6 total, 6 up, 6 in 2023-06-22T18:07:37.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:37.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi133 2023-06-22T18:07:37.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.606 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:37.606 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:37.606 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi133 to 8586M 2023-06-22T18:07:37.607 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.607 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:37.607 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:37.607 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:07:37.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:07:37.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: osd.5 [v2:172.21.15.133:6808/1467210441,v1:172.21.15.133:6809/1467210441] boot 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: osdmap e38: 6 total, 6 up, 6 in 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi133 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:37.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:07:37.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi133 to 8586M 2023-06-22T18:07:37.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:37.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:07:37.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:07:38.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:38 smithi039 ceph-mon[109976]: osdmap e39: 6 total, 6 up, 6 in 2023-06-22T18:07:38.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:38 smithi039 ceph-mon[109976]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:38.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:38 smithi101 ceph-mon[116267]: osdmap e39: 6 total, 6 up, 6 in 2023-06-22T18:07:38.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:38 smithi101 ceph-mon[116267]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:38.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:38 smithi133 ceph-mon[116867]: osdmap e39: 6 total, 6 up, 6 in 2023-06-22T18:07:38.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:38 smithi133 ceph-mon[116867]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:39.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 -- 172.21.15.39:0/4163663452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fee40 msgr2=0x7f8c200ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:39.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/4163663452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fee40 0x7f8c200ff250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c140049c0 tx=0x7f8c14009e30 comp rx=0 tx=0).stop 2023-06-22T18:07:39.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 -- 172.21.15.39:0/4163663452 shutdown_connections 2023-06-22T18:07:39.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/4163663452 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 0x7f8c20107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:39.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/4163663452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fee40 0x7f8c200ff250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:39.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/4163663452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fffb0 0x7f8c20100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:39.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 -- 172.21.15.39:0/4163663452 >> 172.21.15.39:0/4163663452 conn(0x7f8c200fa580 msgr2=0x7f8c200fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:07:39.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 -- 172.21.15.39:0/4163663452 shutdown_connections 2023-06-22T18:07:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.326+0000 7f8c26ed2700 1 -- 172.21.15.39:0/4163663452 wait complete. 2023-06-22T18:07:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 Processor -- start 2023-06-22T18:07:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 -- start start 2023-06-22T18:07:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 0x7f8c2010fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fffb0 0x7f8c20110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 0x7f8c20117100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20117710 con 0x7f8c200fffb0 2023-06-22T18:07:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20117880 con 0x7f8c20100960 2023-06-22T18:07:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c26ed2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20117a40 con 0x7f8c200fee40 2023-06-22T18:07:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c24c6e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 0x7f8c2010fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c24c6e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 0x7f8c2010fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58698/0 (socket says 172.21.15.39:58698) 2023-06-22T18:07:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c1ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fffb0 0x7f8c20110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.327+0000 7f8c24c6e700 1 -- 172.21.15.39:0/3921443515 learned_addr learned my addr 172.21.15.39:0/3921443515 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c2546f700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 0x7f8c20117100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c24c6e700 1 -- 172.21.15.39:0/3921443515 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 msgr2=0x7f8c20117100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c24c6e700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 0x7f8c20117100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c24c6e700 1 -- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fffb0 msgr2=0x7f8c20110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c24c6e700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fffb0 0x7f8c20110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c24c6e700 1 -- 172.21.15.39:0/3921443515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14005040 con 0x7f8c200fee40 2023-06-22T18:07:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c2546f700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 0x7f8c20117100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c24c6e700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 0x7f8c2010fb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009180 tx=0x7f8c14009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14020070 con 0x7f8c200fee40 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c14009520 con 0x7f8c200fee40 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c20117cc0 con 0x7f8c200fee40 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.328+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400e070 con 0x7f8c200fee40 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.329+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c201180c0 con 0x7f8c200fee40 2023-06-22T18:07:39.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.329+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20105d10 con 0x7f8c200fee40 2023-06-22T18:07:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.331+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f8c14009690 con 0x7f8c200fee40 2023-06-22T18:07:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.333+0000 7f8c1d7fa700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8c1009b680 0x7f8c1009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:07:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.333+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4529+0+0 (secure 0 0 0) 0x7f8c140c5450 con 0x7f8c200fee40 2023-06-22T18:07:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.333+0000 7f8c1ffff700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8c1009b680 0x7f8c1009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:07:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.333+0000 7f8c1ffff700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8c1009b680 0x7f8c1009db30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009b80 tx=0x7f8c0c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:07:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.335+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f8c14091ad0 con 0x7f8c200fee40 2023-06-22T18:07:39.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:39 smithi039 ceph-mon[109976]: osdmap e40: 6 total, 6 up, 6 in 2023-06-22T18:07:39.549 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:07:39.546+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c2002b3d0 con 0x7f8c1009b680 2023-06-22T18:07:39.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:39 smithi101 ceph-mon[116267]: osdmap e40: 6 total, 6 up, 6 in 2023-06-22T18:07:39.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:39 smithi133 ceph-mon[116867]: osdmap e40: 6 total, 6 up, 6 in 2023-06-22T18:07:40.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:40 smithi101 ceph-mon[116267]: osdmap e41: 6 total, 6 up, 6 in 2023-06-22T18:07:40.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:40 smithi101 ceph-mon[116267]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:07:40.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:07:40.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:07:40.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:40.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:40 smithi101 ceph-mon[116267]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:40.636 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:40 smithi039 ceph-mon[109976]: osdmap e41: 6 total, 6 up, 6 in 2023-06-22T18:07:40.636 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:40 smithi039 ceph-mon[109976]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:07:40.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:07:40.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:07:40.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:40.637 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:40 smithi039 ceph-mon[109976]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:40 smithi133 ceph-mon[116867]: osdmap e41: 6 total, 6 up, 6 in 2023-06-22T18:07:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:40 smithi133 ceph-mon[116867]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:07:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:07:40.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:07:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:40.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:40 smithi133 ceph-mon[116867]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:41.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:41 smithi101 ceph-mon[116267]: osdmap e42: 6 total, 6 up, 6 in 2023-06-22T18:07:41.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:41 smithi039 ceph-mon[109976]: osdmap e42: 6 total, 6 up, 6 in 2023-06-22T18:07:41.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:41 smithi133 ceph-mon[116867]: osdmap e42: 6 total, 6 up, 6 in 2023-06-22T18:07:42.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:42 smithi101 ceph-mon[116267]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-06-22T18:07:42.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:42 smithi039 ceph-mon[109976]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-06-22T18:07:42.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:42 smithi133 ceph-mon[116867]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-06-22T18:07:43.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:43 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2921574275' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf1482c-6b66-4260-a892-da6969cc8574"}]: dispatch 2023-06-22T18:07:43.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:43 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2921574275' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf1482c-6b66-4260-a892-da6969cc8574"}]': finished 2023-06-22T18:07:43.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:43 smithi101 ceph-mon[116267]: osdmap e43: 7 total, 6 up, 7 in 2023-06-22T18:07:43.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:43 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:07:43.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:43 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2921574275' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf1482c-6b66-4260-a892-da6969cc8574"}]: dispatch 2023-06-22T18:07:43.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:43 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2921574275' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf1482c-6b66-4260-a892-da6969cc8574"}]': finished 2023-06-22T18:07:43.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:43 smithi039 ceph-mon[109976]: osdmap e43: 7 total, 6 up, 7 in 2023-06-22T18:07:43.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:43 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:07:43.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:43 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2921574275' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf1482c-6b66-4260-a892-da6969cc8574"}]: dispatch 2023-06-22T18:07:43.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:43 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2921574275' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf1482c-6b66-4260-a892-da6969cc8574"}]': finished 2023-06-22T18:07:43.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:43 smithi133 ceph-mon[116867]: osdmap e43: 7 total, 6 up, 7 in 2023-06-22T18:07:43.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:43 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:07:44.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:44 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2402270136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:07:44.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:44 smithi101 ceph-mon[116267]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-06-22T18:07:44.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:44 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2402270136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:07:44.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:44 smithi039 ceph-mon[109976]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-06-22T18:07:44.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:44 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2402270136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:07:44.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:44 smithi133 ceph-mon[116867]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2023-06-22T18:07:46.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:46 smithi039 ceph-mon[109976]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-06-22T18:07:47.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:46 smithi101 ceph-mon[116267]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-06-22T18:07:47.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:46 smithi133 ceph-mon[116867]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-06-22T18:07:49.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:48 smithi101 ceph-mon[116267]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-22T18:07:49.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:48 smithi039 ceph-mon[109976]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-22T18:07:49.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:48 smithi133 ceph-mon[116867]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-22T18:07:51.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:50 smithi039 ceph-mon[109976]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:51.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-22T18:07:51.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:51.013 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:50 smithi039 ceph-mon[109976]: Deploying daemon osd.6 on smithi039 2023-06-22T18:07:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:50 smithi101 ceph-mon[116267]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-22T18:07:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:51.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:50 smithi101 ceph-mon[116267]: Deploying daemon osd.6 on smithi039 2023-06-22T18:07:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:50 smithi133 ceph-mon[116867]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-22T18:07:51.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:07:51.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:50 smithi133 ceph-mon[116867]: Deploying daemon osd.6 on smithi039 2023-06-22T18:07:53.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:52 smithi039 ceph-mon[109976]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:53.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:52 smithi101 ceph-mon[116267]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:52 smithi133 ceph-mon[116867]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:55.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:54 smithi101 ceph-mon[116267]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:55.174 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:54 smithi039 ceph-mon[109976]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:55.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:54 smithi133 ceph-mon[116867]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:57.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:56 smithi101 ceph-mon[116267]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:57.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:57.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:57.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:57.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:56 smithi039 ceph-mon[109976]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:56 smithi133 ceph-mon[116867]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:07:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:07:59.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:07:58 smithi101 ceph-mon[116267]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:59.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:07:58 smithi039 ceph-mon[109976]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:07:59.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:07:58 smithi133 ceph-mon[116867]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:00.065 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2023-06-22T18:08:00.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.061+0000 7f8c1d7fa700 1 -- 172.21.15.39:0/3921443515 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c2002b3d0 con 0x7f8c1009b680 2023-06-22T18:08:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.063+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8c1009b680 msgr2=0x7f8c1009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.063+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8c1009b680 0x7f8c1009db30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009b80 tx=0x7f8c0c00b040 comp rx=0 tx=0).stop 2023-06-22T18:08:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.063+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 msgr2=0x7f8c2010fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:00.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.063+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 0x7f8c2010fb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009180 tx=0x7f8c14009e30 comp rx=0 tx=0).stop 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 shutdown_connections 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f8c1009b680 0x7f8c1009db30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c20100960 0x7f8c20117100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c200fee40 0x7f8c2010fb30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 --2- 172.21.15.39:0/3921443515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c200fffb0 0x7f8c20110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 >> 172.21.15.39:0/3921443515 conn(0x7f8c200fa580 msgr2=0x7f8c200fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 shutdown_connections 2023-06-22T18:08:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:00.064+0000 7f8c26ed2700 1 -- 172.21.15.39:0/3921443515 wait complete. 2023-06-22T18:08:00.535 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:00.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:00.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:00.537 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.538 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.538 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:00.588 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.6.service 2023-06-22T18:08:00.593 INFO:tasks.cephadm:Deploying osd.7 on smithi101 with /dev/vg_nvme/lv_2... 2023-06-22T18:08:00.593 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-22T18:08:00.660 INFO:journalctl@ceph.osd.6.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:08:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:00.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:00.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:01.179 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:08:01.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:01 smithi039 ceph-mon[109976]: from='osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-22T18:08:01.554 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:01 smithi039 ceph-mon[109976]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-22T18:08:01.652 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:01 smithi101 ceph-mon[116267]: from='osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-22T18:08:01.653 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:01 smithi101 ceph-mon[116267]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-22T18:08:01.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:01 smithi133 ceph-mon[116867]: from='osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-22T18:08:01.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:01 smithi133 ceph-mon[116867]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-22T18:08:02.598 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:02 smithi101 ceph-mon[116267]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-22T18:08:02.598 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:02 smithi101 ceph-mon[116267]: osdmap e44: 7 total, 6 up, 7 in 2023-06-22T18:08:02.598 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:02 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:02.598 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:02 smithi101 ceph-mon[116267]: from='osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:08:02.599 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:02 smithi101 ceph-mon[116267]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:08:02.599 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:02 smithi101 ceph-mon[116267]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:02.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:02 smithi039 ceph-mon[109976]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-22T18:08:02.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:02 smithi039 ceph-mon[109976]: osdmap e44: 7 total, 6 up, 7 in 2023-06-22T18:08:02.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:02 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:02.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:02 smithi039 ceph-mon[109976]: from='osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:08:02.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:02 smithi039 ceph-mon[109976]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:08:02.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:02 smithi039 ceph-mon[109976]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:02.946 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:08:02 smithi039 conmon[140288]: 2023-06-22T18:08:02.485+0000 7f2c74fa0700 -1 osd.6 0 waiting for initial osdmap 2023-06-22T18:08:02.947 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:08:02 smithi039 conmon[140288]: 2023-06-22T18:08:02.498+0000 7f2c70d9d700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:08:02.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:02 smithi133 ceph-mon[116867]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-22T18:08:02.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:02 smithi133 ceph-mon[116867]: osdmap e44: 7 total, 6 up, 7 in 2023-06-22T18:08:02.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:02 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:02.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:02 smithi133 ceph-mon[116867]: from='osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:08:02.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:02 smithi133 ceph-mon[116867]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:08:02.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:02 smithi133 ceph-mon[116867]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:03.070 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-22T18:08:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-22T18:08:03.071 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-06-22T18:08:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-06-22T18:08:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116496 s, 900 MB/s 2023-06-22T18:08:03.071 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-06-22T18:08:03.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: osdmap e45: 7 total, 6 up, 7 in 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:03.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:03 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.810 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:08:03.810 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:08:03.810 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:08:03.810 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: osdmap e45: 7 total, 6 up, 7 in 2023-06-22T18:08:03.810 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.811 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.812 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:03.812 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:03.812 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:03 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.813 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_2 2023-06-22T18:08:03.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:08:03.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:08:03.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:08:03.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: osdmap e45: 7 total, 6 up, 7 in 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:03.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:03.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:03.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:03.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:03 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:04.389 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:08:04.755 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:04 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi039 2023-06-22T18:08:04.755 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:04 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi039 to 5755M 2023-06-22T18:08:04.756 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:04 smithi101 ceph-mon[116267]: osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120] boot 2023-06-22T18:08:04.756 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:04 smithi101 ceph-mon[116267]: osdmap e46: 7 total, 7 up, 7 in 2023-06-22T18:08:04.756 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:04.756 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:04 smithi101 ceph-mon[116267]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:04.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:04 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi039 2023-06-22T18:08:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:04 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi039 to 5755M 2023-06-22T18:08:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:04 smithi039 ceph-mon[109976]: osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120] boot 2023-06-22T18:08:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:04 smithi039 ceph-mon[109976]: osdmap e46: 7 total, 7 up, 7 in 2023-06-22T18:08:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:04.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:04 smithi039 ceph-mon[109976]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:04 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi039 2023-06-22T18:08:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:04 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi039 to 5755M 2023-06-22T18:08:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:04 smithi133 ceph-mon[116867]: osd.6 [v2:172.21.15.39:6818/4262819120,v1:172.21.15.39:6819/4262819120] boot 2023-06-22T18:08:04.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:04 smithi133 ceph-mon[116867]: osdmap e46: 7 total, 7 up, 7 in 2023-06-22T18:08:04.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-22T18:08:04.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:04 smithi133 ceph-mon[116867]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-06-22T18:08:05.778 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:05 smithi101 ceph-mon[116267]: osdmap e47: 7 total, 7 up, 7 in 2023-06-22T18:08:05.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:05 smithi039 ceph-mon[109976]: osdmap e47: 7 total, 7 up, 7 in 2023-06-22T18:08:05.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:05 smithi133 ceph-mon[116867]: osdmap e47: 7 total, 7 up, 7 in 2023-06-22T18:08:06.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 -- 172.21.15.101:0/1616144159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fcd20 msgr2=0x7fea940fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:06.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/1616144159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fcd20 0x7fea940fd170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea840049c0 tx=0x7fea84009e30 comp rx=0 tx=0).stop 2023-06-22T18:08:06.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 -- 172.21.15.101:0/1616144159 shutdown_connections 2023-06-22T18:08:06.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/1616144159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea940fd6b0 0x7fea94103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:06.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/1616144159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea940fbb10 0x7fea940fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:06.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/1616144159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fcd20 0x7fea940fd170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:06.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 -- 172.21.15.101:0/1616144159 >> 172.21.15.101:0/1616144159 conn(0x7fea940f7270 msgr2=0x7fea940f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:08:06.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 -- 172.21.15.101:0/1616144159 shutdown_connections 2023-06-22T18:08:06.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.001+0000 7fea9bdcc700 1 -- 172.21.15.101:0/1616144159 wait complete. 2023-06-22T18:08:06.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 Processor -- start 2023-06-22T18:08:06.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 -- start start 2023-06-22T18:08:06.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 0x7fea94066a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:06.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea940fcd20 0x7fea94114c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:06.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea940fd6b0 0x7fea941171a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:06.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea94117860 con 0x7fea940fcd20 2023-06-22T18:08:06.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea941179d0 con 0x7fea940fbb10 2023-06-22T18:08:06.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea9bdcc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea94117b90 con 0x7fea940fd6b0 2023-06-22T18:08:06.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 0x7fea94066a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 0x7fea94066a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44216/0 (socket says 172.21.15.101:44216) 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 -- 172.21.15.101:0/840744667 learned_addr learned my addr 172.21.15.101:0/840744667 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 -- 172.21.15.101:0/840744667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea940fd6b0 msgr2=0x7fea941171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea940fd6b0 0x7fea941171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 -- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea940fcd20 msgr2=0x7fea94114c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea940fcd20 0x7fea94114c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 -- 172.21.15.101:0/840744667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84005040 con 0x7fea940fbb10 2023-06-22T18:08:06.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.002+0000 7fea99b68700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 0x7fea94066a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea9000b810 tx=0x7fea90009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.003+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9000f5f0 con 0x7fea940fbb10 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.003+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea9000fc20 con 0x7fea940fbb10 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.003+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea94117e10 con 0x7fea940fbb10 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.003+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90010cc0 con 0x7fea940fbb10 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.003+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea94118290 con 0x7fea940fbb10 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.004+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea94107420 con 0x7fea940fbb10 2023-06-22T18:08:06.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.005+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fea9000fd90 con 0x7fea940fbb10 2023-06-22T18:08:06.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.005+0000 7fea8a7fc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fea8009b700 0x7fea8009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:06.011 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.005+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4961+0+0 (secure 0 0 0) 0x7fea900c4020 con 0x7fea940fbb10 2023-06-22T18:08:06.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.005+0000 7fea99367700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fea8009b700 0x7fea8009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:06.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.006+0000 7fea99367700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fea8009b700 0x7fea8009dbb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea84005010 tx=0x7fea84009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:08:06.012 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.009+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fea90090590 con 0x7fea940fbb10 2023-06-22T18:08:06.202 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:06.199+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fea9405de70 con 0x7fea8009b700 2023-06-22T18:08:06.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:06 smithi101 ceph-mon[116267]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:06.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:06.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:06.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:06 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:06.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:06 smithi039 ceph-mon[109976]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:06.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:06.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:06.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:06 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:06.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:06 smithi133 ceph-mon[116867]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:06.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:06.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:06.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:06 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:07.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:07 smithi101 ceph-mon[116267]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:07.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:07 smithi039 ceph-mon[109976]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:07.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:07 smithi133 ceph-mon[116867]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:08.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:08 smithi101 ceph-mon[116267]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:08.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:08 smithi039 ceph-mon[109976]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:08.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:08 smithi133 ceph-mon[116867]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:09.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:09 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/1350438463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]: dispatch 2023-06-22T18:08:09.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:09 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]: dispatch 2023-06-22T18:08:09.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:09 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]': finished 2023-06-22T18:08:09.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:09 smithi101 ceph-mon[116267]: osdmap e48: 8 total, 7 up, 8 in 2023-06-22T18:08:09.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:09 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:09.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:09 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/3766814634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:08:09.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:09 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/1350438463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]: dispatch 2023-06-22T18:08:09.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:09 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]: dispatch 2023-06-22T18:08:09.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:09 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]': finished 2023-06-22T18:08:09.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:09 smithi039 ceph-mon[109976]: osdmap e48: 8 total, 7 up, 8 in 2023-06-22T18:08:09.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:09.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:09 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/3766814634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:08:09.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:09 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/1350438463' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]: dispatch 2023-06-22T18:08:09.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:09 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]: dispatch 2023-06-22T18:08:09.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:09 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b535b3c-1d36-44c9-8c22-132e6d31b0ea"}]': finished 2023-06-22T18:08:09.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:09 smithi133 ceph-mon[116867]: osdmap e48: 8 total, 7 up, 8 in 2023-06-22T18:08:09.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:09 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:09.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:09 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/3766814634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:08:11.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:10 smithi101 ceph-mon[116267]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:11.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:10 smithi039 ceph-mon[109976]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:11.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:10 smithi133 ceph-mon[116867]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:13.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:12 smithi101 ceph-mon[116267]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:13.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:12 smithi039 ceph-mon[109976]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:13.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:12 smithi133 ceph-mon[116867]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:15.101 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:14 smithi101 ceph-mon[116267]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:15.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:14 smithi039 ceph-mon[109976]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:15.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:14 smithi133 ceph-mon[116867]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:16 smithi101 ceph-mon[116267]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-22T18:08:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:17.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:16 smithi039 ceph-mon[109976]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:17.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-22T18:08:17.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:16 smithi133 ceph-mon[116867]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-22T18:08:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:18.009 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:17 smithi101 ceph-mon[116267]: Deploying daemon osd.7 on smithi101 2023-06-22T18:08:18.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:17 smithi039 ceph-mon[109976]: Deploying daemon osd.7 on smithi101 2023-06-22T18:08:18.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:17 smithi133 ceph-mon[116867]: Deploying daemon osd.7 on smithi101 2023-06-22T18:08:19.031 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:18 smithi101 ceph-mon[116267]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:19.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:18 smithi039 ceph-mon[109976]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:19.233 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:18 smithi133 ceph-mon[116867]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:21.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:20 smithi101 ceph-mon[116267]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:21.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:20 smithi039 ceph-mon[109976]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:21.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:20 smithi133 ceph-mon[116867]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:22.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:22 smithi101 ceph-mon[116267]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:22.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:22.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:22.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:22 smithi039 ceph-mon[109976]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:22 smithi133 ceph-mon[116867]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:25.098 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:24 smithi101 ceph-mon[116267]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:25.098 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.098 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.098 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:25.098 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:25.099 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:24 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.175 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.171+0000 7fea8a7fc700 1 -- 172.21.15.101:0/840744667 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea9405de70 con 0x7fea8009b700 2023-06-22T18:08:25.176 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 7 on host 'smithi101' 2023-06-22T18:08:25.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fea8009b700 msgr2=0x7fea8009dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:25.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fea8009b700 0x7fea8009dbb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea84005010 tx=0x7fea84009e30 comp rx=0 tx=0).stop 2023-06-22T18:08:25.177 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 msgr2=0x7fea94066a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 0x7fea94066a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea9000b810 tx=0x7fea90009220 comp rx=0 tx=0).stop 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 shutdown_connections 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fea8009b700 0x7fea8009dbb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea940fd6b0 0x7fea941171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fea940fbb10 0x7fea94066a50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 --2- 172.21.15.101:0/840744667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea940fcd20 0x7fea94114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:25.178 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 >> 172.21.15.101:0/840744667 conn(0x7fea940f7270 msgr2=0x7fea940f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:08:25.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 shutdown_connections 2023-06-22T18:08:25.179 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:08:25.173+0000 7fea9bdcc700 1 -- 172.21.15.101:0/840744667 wait complete. 2023-06-22T18:08:25.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:24 smithi039 ceph-mon[109976]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:25.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:25.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:25.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:24 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:24 smithi133 ceph-mon[116867]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:25.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:25.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:25.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:24 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:25.842 DEBUG:teuthology.orchestra.run.smithi101:osd.7> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.7.service 2023-06-22T18:08:25.845 INFO:tasks.cephadm:Deploying osd.8 on smithi133 with /dev/vg_nvme/lv_2... 2023-06-22T18:08:25.845 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-22T18:08:25.871 INFO:journalctl@ceph.osd.7.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:08:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:26 smithi133 ceph-mon[116867]: from='osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-22T18:08:26.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:26 smithi133 ceph-mon[116867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-22T18:08:26.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:26 smithi133 ceph-mon[116867]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:26.437 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:08:26.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:26.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:26.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:26.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:26 smithi039 ceph-mon[109976]: from='osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-22T18:08:26.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:26 smithi039 ceph-mon[109976]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-22T18:08:26.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:26 smithi039 ceph-mon[109976]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:26.497 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:26.497 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:26.497 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:26.497 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:26 smithi101 ceph-mon[116267]: from='osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-22T18:08:26.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:26 smithi101 ceph-mon[116267]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-22T18:08:26.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:26 smithi101 ceph-mon[116267]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:27 smithi133 ceph-mon[116867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-22T18:08:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:27 smithi133 ceph-mon[116867]: osdmap e49: 8 total, 7 up, 8 in 2023-06-22T18:08:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:27 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:27 smithi133 ceph-mon[116867]: from='osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:08:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:27 smithi133 ceph-mon[116867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:08:27.419 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:27 smithi101 ceph-mon[116267]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-22T18:08:27.420 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:27 smithi101 ceph-mon[116267]: osdmap e49: 8 total, 7 up, 8 in 2023-06-22T18:08:27.420 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:27 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:27.420 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:27 smithi101 ceph-mon[116267]: from='osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:08:27.420 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:27 smithi101 ceph-mon[116267]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:08:27.420 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:08:27 smithi101 conmon[136691]: 2023-06-22T18:08:27.142+0000 7f9c24beb700 -1 osd.7 0 waiting for initial osdmap 2023-06-22T18:08:27.421 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:08:27 smithi101 conmon[136691]: 2023-06-22T18:08:27.152+0000 7f9c1e9e4700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:08:27.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:27 smithi039 ceph-mon[109976]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-22T18:08:27.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:27 smithi039 ceph-mon[109976]: osdmap e49: 8 total, 7 up, 8 in 2023-06-22T18:08:27.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:27 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:27.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:27 smithi039 ceph-mon[109976]: from='osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:08:27.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:27 smithi039 ceph-mon[109976]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:08:28.103 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-22T18:08:28.104 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-22T18:08:28.104 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-06-22T18:08:28.104 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-06-22T18:08:28.104 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124479 s, 842 MB/s 2023-06-22T18:08:28.104 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 2023-06-22T18:08:28.105 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-06-22T18:08:28.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:28 smithi101 ceph-mon[116267]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:08:28.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:28 smithi101 ceph-mon[116267]: osdmap e50: 8 total, 7 up, 8 in 2023-06-22T18:08:28.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:28 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:28.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:28 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:28.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:28 smithi101 ceph-mon[116267]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:28.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:28 smithi039 ceph-mon[109976]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:08:28.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:28 smithi039 ceph-mon[109976]: osdmap e50: 8 total, 7 up, 8 in 2023-06-22T18:08:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:28 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:28 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:28.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:28 smithi039 ceph-mon[109976]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:28.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:28 smithi133 ceph-mon[116867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:08:28.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:28 smithi133 ceph-mon[116867]: osdmap e50: 8 total, 7 up, 8 in 2023-06-22T18:08:28.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:28 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:28.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:28 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:28.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:28 smithi133 ceph-mon[116867]: pgmap v177: 1 pgs: 1 peering; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-06-22T18:08:28.768 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2023-06-22T18:08:29.350 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:08:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401] boot 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: osdmap e51: 8 total, 8 up, 8 in 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi101 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi101 to 5724M 2023-06-22T18:08:29.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:29.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:08:29.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:08:29.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-06-22T18:08:29.487 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:29.487 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401] boot 2023-06-22T18:08:29.487 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: osdmap e51: 8 total, 8 up, 8 in 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi101 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.488 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.489 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi101 to 5724M 2023-06-22T18:08:29.489 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.489 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:29.489 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:29.490 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:08:29.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:08:29.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-06-22T18:08:29.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:29.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: osd.7 [v2:172.21.15.101:6816/3667650401,v1:172.21.15.101:6817/3667650401] boot 2023-06-22T18:08:29.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: osdmap e51: 8 total, 8 up, 8 in 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi101 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:29.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi101 to 5724M 2023-06-22T18:08:29.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:29.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:29.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:29.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:30.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:30 smithi039 ceph-mon[109976]: osdmap e52: 8 total, 8 up, 8 in 2023-06-22T18:08:30.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:30 smithi039 ceph-mon[109976]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:30.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:30 smithi133 ceph-mon[116867]: osdmap e52: 8 total, 8 up, 8 in 2023-06-22T18:08:30.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:30 smithi133 ceph-mon[116867]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:30.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:30 smithi101 ceph-mon[116267]: osdmap e52: 8 total, 8 up, 8 in 2023-06-22T18:08:30.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:30 smithi101 ceph-mon[116267]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:30.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- 172.21.15.133:0/1587311353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fde90 msgr2=0x7f2dfc0fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:30.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1587311353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc0fe2e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2dec0049c0 tx=0x7f2dec009e30 comp rx=0 tx=0).stop 2023-06-22T18:08:30.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- 172.21.15.133:0/1587311353 shutdown_connections 2023-06-22T18:08:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1587311353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1587311353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dfc0fcc80 0x7f2dfc0fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1587311353 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc0fe2e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- 172.21.15.133:0/1587311353 >> 172.21.15.133:0/1587311353 conn(0x7f2dfc0f83c0 msgr2=0x7f2dfc0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:08:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- 172.21.15.133:0/1587311353 shutdown_connections 2023-06-22T18:08:30.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- 172.21.15.133:0/1587311353 wait complete. 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 Processor -- start 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- start start 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dfc0fcc80 0x7f2dfc10d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc114e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc115450 con 0x7f2dfc0fcc80 2023-06-22T18:08:30.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc1155c0 con 0x7f2dfc0fde90 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e0337b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc115780 con 0x7f2dfc0fe820 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e01918700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc114e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e01918700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc114e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47690/0 (socket says 172.21.15.133:47690) 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.908+0000 7f2e01918700 1 -- 172.21.15.133:0/1689954447 learned_addr learned my addr 172.21.15.133:0/1689954447 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01117700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dfc0fcc80 0x7f2dfc10d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e00916700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:30.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01918700 1 -- 172.21.15.133:0/1689954447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fde90 msgr2=0x7f2dfc10ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01918700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc10ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01918700 1 -- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dfc0fcc80 msgr2=0x7f2dfc10d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01918700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dfc0fcc80 0x7f2dfc10d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01918700 1 -- 172.21.15.133:0/1689954447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec005040 con 0x7f2dfc0fe820 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e00916700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e01918700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc114e40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2df400cb90 tx=0x7f2df4007ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400a870 con 0x7f2dfc0fe820 2023-06-22T18:08:30.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2df4012440 con 0x7f2dfc0fe820 2023-06-22T18:08:30.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc115a60 con 0x7f2dfc0fe820 2023-06-22T18:08:30.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400aea0 con 0x7f2dfc0fe820 2023-06-22T18:08:30.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.909+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc115fd0 con 0x7f2dfc0fe820 2023-06-22T18:08:30.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.910+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc108420 con 0x7f2dfc0fe820 2023-06-22T18:08:30.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.911+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f2df40125b0 con 0x7f2dfc0fe820 2023-06-22T18:08:30.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.912+0000 7f2df1ffb700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2de809b780 0x7f2de809dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:30.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.912+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5422+0+0 (secure 0 0 0) 0x7f2df40c4f30 con 0x7f2dfc0fe820 2023-06-22T18:08:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.912+0000 7f2e01117700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2de809b780 0x7f2de809dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.913+0000 7f2e01117700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2de809b780 0x7f2de809dc30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2df8007fd0 tx=0x7f2df8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:08:30.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:30.915+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f2df4091240 con 0x7f2dfc0fe820 2023-06-22T18:08:31.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:31.106+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dfc05de70 con 0x7f2de809b780 2023-06-22T18:08:31.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:31 smithi039 ceph-mon[109976]: osdmap e53: 8 total, 8 up, 8 in 2023-06-22T18:08:31.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:31 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:31.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:31 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:31.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:31 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:31.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:31 smithi133 ceph-mon[116867]: osdmap e53: 8 total, 8 up, 8 in 2023-06-22T18:08:31.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:31 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:31.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:31 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:31.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:31 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:31.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:31 smithi101 ceph-mon[116267]: osdmap e53: 8 total, 8 up, 8 in 2023-06-22T18:08:31.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:31 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:31.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:31 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:31.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:31 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:32.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:32 smithi133 ceph-mon[116867]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:32.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:32 smithi133 ceph-mon[116867]: osdmap e54: 8 total, 8 up, 8 in 2023-06-22T18:08:32.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:32 smithi133 ceph-mon[116867]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:32.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:32 smithi101 ceph-mon[116267]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:32.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:32 smithi101 ceph-mon[116267]: osdmap e54: 8 total, 8 up, 8 in 2023-06-22T18:08:32.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:32 smithi101 ceph-mon[116267]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:32.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:32 smithi039 ceph-mon[109976]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:32.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:32 smithi039 ceph-mon[109976]: osdmap e54: 8 total, 8 up, 8 in 2023-06-22T18:08:32.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:32 smithi039 ceph-mon[109976]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:33.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:33 smithi133 ceph-mon[116867]: osdmap e55: 8 total, 8 up, 8 in 2023-06-22T18:08:33.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:33 smithi101 ceph-mon[116267]: osdmap e55: 8 total, 8 up, 8 in 2023-06-22T18:08:33.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:33 smithi039 ceph-mon[109976]: osdmap e55: 8 total, 8 up, 8 in 2023-06-22T18:08:34.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:34 smithi133 ceph-mon[116867]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:34.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:34 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/698908744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]: dispatch 2023-06-22T18:08:34.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:34 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]: dispatch 2023-06-22T18:08:34.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:34 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]': finished 2023-06-22T18:08:34.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:34 smithi133 ceph-mon[116867]: osdmap e56: 9 total, 8 up, 9 in 2023-06-22T18:08:34.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:34 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:34.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:34 smithi101 ceph-mon[116267]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:34 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/698908744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]: dispatch 2023-06-22T18:08:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:34 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]: dispatch 2023-06-22T18:08:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:34 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]': finished 2023-06-22T18:08:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:34 smithi101 ceph-mon[116267]: osdmap e56: 9 total, 8 up, 9 in 2023-06-22T18:08:34.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:34 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:34.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:34 smithi039 ceph-mon[109976]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:34.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:34 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/698908744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]: dispatch 2023-06-22T18:08:34.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:34 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]: dispatch 2023-06-22T18:08:34.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:34 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60376342-edba-4eb8-9907-950963e6c3e0"}]': finished 2023-06-22T18:08:34.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:34 smithi039 ceph-mon[109976]: osdmap e56: 9 total, 8 up, 9 in 2023-06-22T18:08:34.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:34 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:35.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:35 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/970372650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:08:35.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:35 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/970372650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:08:35.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:35 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/970372650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:08:36.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:36 smithi133 ceph-mon[116867]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:36.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:36 smithi101 ceph-mon[116267]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:36.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:36 smithi039 ceph-mon[109976]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:37.483 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:37 smithi133 ceph-mon[116867]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-06-22T18:08:37.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:37 smithi133 ceph-mon[116867]: Cluster is now healthy 2023-06-22T18:08:37.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:37 smithi101 ceph-mon[116267]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-06-22T18:08:37.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:37 smithi101 ceph-mon[116267]: Cluster is now healthy 2023-06-22T18:08:37.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:37 smithi039 ceph-mon[109976]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-06-22T18:08:37.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:37 smithi039 ceph-mon[109976]: Cluster is now healthy 2023-06-22T18:08:38.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:38 smithi133 ceph-mon[116867]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:38.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:38 smithi101 ceph-mon[116267]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:38.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:38 smithi039 ceph-mon[109976]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:40.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:40 smithi133 ceph-mon[116867]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:41.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:40 smithi101 ceph-mon[116267]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:41.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:40 smithi039 ceph-mon[109976]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:42.053 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:41 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-22T18:08:42.054 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:41 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:42.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:41 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-22T18:08:42.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:41 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:42.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:41 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-22T18:08:42.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:41 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:43.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:42 smithi133 ceph-mon[116867]: Deploying daemon osd.8 on smithi133 2023-06-22T18:08:43.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:42 smithi133 ceph-mon[116867]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:43.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:42 smithi101 ceph-mon[116267]: Deploying daemon osd.8 on smithi133 2023-06-22T18:08:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:42 smithi101 ceph-mon[116267]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:42 smithi039 ceph-mon[109976]: Deploying daemon osd.8 on smithi133 2023-06-22T18:08:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:42 smithi039 ceph-mon[109976]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:45.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:44 smithi101 ceph-mon[116267]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:44 smithi039 ceph-mon[109976]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:44 smithi133 ceph-mon[116867]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:47.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:46 smithi101 ceph-mon[116267]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:47.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:47.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:46 smithi039 ceph-mon[109976]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:47.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:47.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:47.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:47.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:46 smithi133 ceph-mon[116867]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:47.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:47.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:47.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:49.065 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:48 smithi133 ceph-mon[116867]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:49.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:48 smithi101 ceph-mon[116267]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:49.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:48 smithi039 ceph-mon[109976]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:50.285 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 8 on host 'smithi133' 2023-06-22T18:08:50.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.282+0000 7f2df1ffb700 1 -- 172.21.15.133:0/1689954447 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2dfc05de70 con 0x7f2de809b780 2023-06-22T18:08:50.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.284+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2de809b780 msgr2=0x7f2de809dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.284+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2de809b780 0x7f2de809dc30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2df8007fd0 tx=0x7f2df8009040 comp rx=0 tx=0).stop 2023-06-22T18:08:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.284+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 msgr2=0x7f2dfc114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.284+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc114e40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2df400cb90 tx=0x7f2df4007ed0 comp rx=0 tx=0).stop 2023-06-22T18:08:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 shutdown_connections 2023-06-22T18:08:50.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2de809b780 0x7f2de809dc30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dfc0fe820 0x7f2dfc114e40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dfc0fcc80 0x7f2dfc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 --2- 172.21.15.133:0/1689954447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2dfc0fde90 0x7f2dfc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 >> 172.21.15.133:0/1689954447 conn(0x7f2dfc0f83c0 msgr2=0x7f2dfc0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:08:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 shutdown_connections 2023-06-22T18:08:50.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:08:50.285+0000 7f2e0337b700 1 -- 172.21.15.133:0/1689954447 wait complete. 2023-06-22T18:08:50.467 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.467 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.467 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:50.467 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:50.467 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.467 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:50.468 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.468 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:50.468 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:50.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:08:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:51.003 DEBUG:teuthology.orchestra.run.smithi133:osd.8> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.8.service 2023-06-22T18:08:51.006 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2023-06-22T18:08:51.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-22T18:08:51.033 INFO:journalctl@ceph.osd.8.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:08:51.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:51 smithi133 ceph-mon[116867]: from='osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-22T18:08:51.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:51 smithi133 ceph-mon[116867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-22T18:08:51.790 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:08:51.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:51 smithi101 ceph-mon[116267]: from='osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-22T18:08:51.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:51 smithi101 ceph-mon[116267]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-22T18:08:51.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:51 smithi039 ceph-mon[109976]: from='osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-22T18:08:51.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:51 smithi039 ceph-mon[109976]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-22T18:08:52.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:52 smithi039 ceph-mon[109976]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-22T18:08:52.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:52 smithi039 ceph-mon[109976]: osdmap e57: 9 total, 8 up, 9 in 2023-06-22T18:08:52.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:52 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:52.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:52 smithi039 ceph-mon[109976]: from='osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:08:52.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:52 smithi039 ceph-mon[109976]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:08:52.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:52 smithi039 ceph-mon[109976]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:52.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:52 smithi133 ceph-mon[116867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-22T18:08:52.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:52 smithi133 ceph-mon[116867]: osdmap e57: 9 total, 8 up, 9 in 2023-06-22T18:08:52.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:52 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:52.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:52 smithi133 ceph-mon[116867]: from='osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:08:52.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:52 smithi133 ceph-mon[116867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:08:52.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:52 smithi133 ceph-mon[116867]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:52.737 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:08:52 smithi133 conmon[138513]: 2023-06-22T18:08:52.479+0000 7f6233e6c700 -1 osd.8 0 waiting for initial osdmap 2023-06-22T18:08:52.737 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:08:52 smithi133 conmon[138513]: 2023-06-22T18:08:52.494+0000 7f622cc63700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:08:52.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:52 smithi101 ceph-mon[116267]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-22T18:08:52.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:52 smithi101 ceph-mon[116267]: osdmap e57: 9 total, 8 up, 9 in 2023-06-22T18:08:52.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:52 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:52.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:52 smithi101 ceph-mon[116267]: from='osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:08:52.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:52 smithi101 ceph-mon[116267]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:08:52.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:52 smithi101 ceph-mon[116267]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-06-22T18:08:53.659 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-22T18:08:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-22T18:08:53.660 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2023-06-22T18:08:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2023-06-22T18:08:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0290796 s, 361 MB/s 2023-06-22T18:08:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2023-06-22T18:08:53.760 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:08:53.761 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: osdmap e58: 9 total, 8 up, 9 in 2023-06-22T18:08:53.761 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.761 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.761 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117] boot 2023-06-22T18:08:53.761 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: osdmap e59: 9 total, 9 up, 9 in 2023-06-22T18:08:53.761 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:53 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:08:53.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: osdmap e58: 9 total, 8 up, 9 in 2023-06-22T18:08:53.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117] boot 2023-06-22T18:08:53.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: osdmap e59: 9 total, 9 up, 9 in 2023-06-22T18:08:53.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:53 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.870 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:08:53.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: osdmap e58: 9 total, 8 up, 9 in 2023-06-22T18:08:53.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:53.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: osd.8 [v2:172.21.15.133:6816/1333554117,v1:172.21.15.133:6817/1333554117] boot 2023-06-22T18:08:53.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: osdmap e59: 9 total, 9 up, 9 in 2023-06-22T18:08:53.871 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:53 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-22T18:08:54.157 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2023-06-22T18:08:54.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:08:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:08:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi133 2023-06-22T18:08:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi133 to 5724M 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:54 smithi101 ceph-mon[116267]: osdmap e60: 9 total, 9 up, 9 in 2023-06-22T18:08:54.926 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:08:54.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:08:54.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi133 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi133 to 5724M 2023-06-22T18:08:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:54 smithi039 ceph-mon[109976]: osdmap e60: 9 total, 9 up, 9 in 2023-06-22T18:08:54.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:08:54.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:08:54.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:54.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi133 2023-06-22T18:08:54.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi133 to 5724M 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:54.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:08:54.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:08:54.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:54 smithi133 ceph-mon[116867]: osdmap e60: 9 total, 9 up, 9 in 2023-06-22T18:08:56.325 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 -- 172.21.15.39:0/175011107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fe7a0 msgr2=0x7fd8c8105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/175011107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c8105060 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009d60 tx=0x7fd8c0004d10 comp rx=0 tx=0).stop 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 -- 172.21.15.39:0/175011107 shutdown_connections 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/175011107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c8105060 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/175011107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8c80fcc00 0x7fd8c80fd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/175011107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fde10 0x7fd8c80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.320+0000 7fd8cfec9700 1 -- 172.21.15.39:0/175011107 >> 172.21.15.39:0/175011107 conn(0x7fd8c80f8320 msgr2=0x7fd8c80fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.321+0000 7fd8cfec9700 1 -- 172.21.15.39:0/175011107 shutdown_connections 2023-06-22T18:08:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.321+0000 7fd8cfec9700 1 -- 172.21.15.39:0/175011107 wait complete. 2023-06-22T18:08:56.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.321+0000 7fd8cfec9700 1 Processor -- start 2023-06-22T18:08:56.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 -- start start 2023-06-22T18:08:56.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 0x7fd8c8071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:56.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8c80fde10 0x7fd8c806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:56.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8070a30 con 0x7fd8c80fcc00 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8070ba0 con 0x7fd8c80fde10 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cfec9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81177f0 con 0x7fd8c80fe7a0 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cdc65700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 0x7fd8c8071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cdc65700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 0x7fd8c8071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51550/0 (socket says 172.21.15.39:51550) 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cdc65700 1 -- 172.21.15.39:0/197561060 learned_addr learned my addr 172.21.15.39:0/197561060 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.322+0000 7fd8cd464700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8c80fde10 0x7fd8c806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8ce466700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cdc65700 1 -- 172.21.15.39:0/197561060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fe7a0 msgr2=0x7fd8c80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cdc65700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cdc65700 1 -- 172.21.15.39:0/197561060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8c80fde10 msgr2=0x7fd8c806fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:08:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cdc65700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8c80fde10 0x7fd8c806fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:08:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cdc65700 1 -- 172.21.15.39:0/197561060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c0009940 con 0x7fd8c80fcc00 2023-06-22T18:08:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8ce466700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cdc65700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 0x7fd8c8071720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400ebf0 tx=0x7fd8c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4007cc0 con 0x7fd8c80fcc00 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c8117a40 con 0x7fd8c80fcc00 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8c400f040 con 0x7fd8c80fcc00 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c400c8f0 con 0x7fd8c80fcc00 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.323+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8117f80 con 0x7fd8c80fcc00 2023-06-22T18:08:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.324+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c81038b0 con 0x7fd8c80fcc00 2023-06-22T18:08:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.326+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fd8c400cd20 con 0x7fd8c80fcc00 2023-06-22T18:08:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.328+0000 7fd8be7fc700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd8b409b700 0x7fd8b409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:08:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.328+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd8c40c3f50 con 0x7fd8c80fcc00 2023-06-22T18:08:56.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.328+0000 7fd8cd464700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd8b409b700 0x7fd8b409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:08:56.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.329+0000 7fd8cd464700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd8b409b700 0x7fd8b409dbb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009b80 tx=0x7fd8b800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:08:56.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.333+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fd8c404da00 con 0x7fd8c80fcc00 2023-06-22T18:08:56.558 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:08:56.554+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8c805de70 con 0x7fd8b409b700 2023-06-22T18:08:56.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:56 smithi101 ceph-mon[116267]: osdmap e61: 9 total, 9 up, 9 in 2023-06-22T18:08:56.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:56 smithi101 ceph-mon[116267]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:56.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:56 smithi039 ceph-mon[109976]: osdmap e61: 9 total, 9 up, 9 in 2023-06-22T18:08:56.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:56 smithi039 ceph-mon[109976]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:56.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:56 smithi133 ceph-mon[116867]: osdmap e61: 9 total, 9 up, 9 in 2023-06-22T18:08:56.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:56 smithi133 ceph-mon[116867]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:57 smithi039 ceph-mon[109976]: osdmap e62: 9 total, 9 up, 9 in 2023-06-22T18:08:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:57 smithi039 ceph-mon[109976]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:57 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:57 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:57.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:57 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:57.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:57 smithi101 ceph-mon[116267]: osdmap e62: 9 total, 9 up, 9 in 2023-06-22T18:08:57.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:57 smithi101 ceph-mon[116267]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:57.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:57 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:57.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:57 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:57.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:57 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:57.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:57 smithi133 ceph-mon[116867]: osdmap e62: 9 total, 9 up, 9 in 2023-06-22T18:08:57.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:57 smithi133 ceph-mon[116867]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:08:57.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:57 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:08:57.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:57 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:08:57.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:57 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:08:58.638 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:58 smithi039 ceph-mon[109976]: osdmap e63: 9 total, 9 up, 9 in 2023-06-22T18:08:58.638 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:58 smithi039 ceph-mon[109976]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:58.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:58 smithi101 ceph-mon[116267]: osdmap e63: 9 total, 9 up, 9 in 2023-06-22T18:08:58.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:58 smithi101 ceph-mon[116267]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:58.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:58 smithi133 ceph-mon[116867]: osdmap e63: 9 total, 9 up, 9 in 2023-06-22T18:08:58.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:58 smithi133 ceph-mon[116867]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:08:59.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:59 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3424440114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c09c5c50-6e36-4e8e-b41c-50fb93a2ea92"}]: dispatch 2023-06-22T18:08:59.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:59 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3424440114' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c09c5c50-6e36-4e8e-b41c-50fb93a2ea92"}]': finished 2023-06-22T18:08:59.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:59 smithi133 ceph-mon[116867]: osdmap e64: 10 total, 9 up, 10 in 2023-06-22T18:08:59.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:08:59 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:00.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:59 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3424440114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c09c5c50-6e36-4e8e-b41c-50fb93a2ea92"}]: dispatch 2023-06-22T18:09:00.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:59 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3424440114' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c09c5c50-6e36-4e8e-b41c-50fb93a2ea92"}]': finished 2023-06-22T18:09:00.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:59 smithi101 ceph-mon[116267]: osdmap e64: 10 total, 9 up, 10 in 2023-06-22T18:09:00.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:08:59 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:59 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3424440114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c09c5c50-6e36-4e8e-b41c-50fb93a2ea92"}]: dispatch 2023-06-22T18:09:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:59 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3424440114' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c09c5c50-6e36-4e8e-b41c-50fb93a2ea92"}]': finished 2023-06-22T18:09:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:59 smithi039 ceph-mon[109976]: osdmap e64: 10 total, 9 up, 10 in 2023-06-22T18:09:00.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:08:59 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:01.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:00 smithi101 ceph-mon[116267]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-06-22T18:09:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:00 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1441880509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:01.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:00 smithi039 ceph-mon[109976]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-06-22T18:09:01.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:00 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1441880509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:01.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:00 smithi133 ceph-mon[116867]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-06-22T18:09:01.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:00 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1441880509' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:03.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:02 smithi101 ceph-mon[116267]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-22T18:09:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:02 smithi039 ceph-mon[109976]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-22T18:09:03.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:02 smithi133 ceph-mon[116867]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-22T18:09:05.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:04 smithi101 ceph-mon[116267]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-06-22T18:09:05.155 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:04 smithi039 ceph-mon[109976]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-06-22T18:09:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:04 smithi133 ceph-mon[116867]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-06-22T18:09:07.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:06 smithi039 ceph-mon[109976]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-06-22T18:09:07.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:06 smithi101 ceph-mon[116267]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-06-22T18:09:07.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:06 smithi133 ceph-mon[116867]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-06-22T18:09:08.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:07 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-22T18:09:08.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:07 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:08.131 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:07 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-22T18:09:08.131 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:07 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:08.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:07 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-22T18:09:08.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:07 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:09.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:08 smithi039 ceph-mon[109976]: Deploying daemon osd.9 on smithi039 2023-06-22T18:09:09.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:08 smithi039 ceph-mon[109976]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-22T18:09:09.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:08 smithi101 ceph-mon[116267]: Deploying daemon osd.9 on smithi039 2023-06-22T18:09:09.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:08 smithi101 ceph-mon[116267]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-22T18:09:09.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:08 smithi133 ceph-mon[116867]: Deploying daemon osd.9 on smithi039 2023-06-22T18:09:09.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:08 smithi133 ceph-mon[116867]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-22T18:09:11.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:10 smithi101 ceph-mon[116267]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:11.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:10 smithi039 ceph-mon[109976]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:11.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:10 smithi133 ceph-mon[116867]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:13.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:12 smithi101 ceph-mon[116267]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:13.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:12 smithi039 ceph-mon[109976]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:13.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:12 smithi133 ceph-mon[116867]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:14.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:14.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:14.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:14 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:14.816 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:14 smithi039 ceph-mon[109976]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:14.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:14 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:14.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:14 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:14.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:14 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:14.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:14 smithi101 ceph-mon[116267]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:14.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:14 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:14.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:14 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:14.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:14 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:14.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:14 smithi133 ceph-mon[116867]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:16 smithi039 ceph-mon[109976]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:16.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:16.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:16 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.064+0000 7fd8be7fc700 1 -- 172.21.15.39:0/197561060 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd8c805de70 con 0x7fd8b409b700 2023-06-22T18:09:17.068 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2023-06-22T18:09:17.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.066+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd8b409b700 msgr2=0x7fd8b409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:17.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.066+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd8b409b700 0x7fd8b409dbb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009b80 tx=0x7fd8b800b040 comp rx=0 tx=0).stop 2023-06-22T18:09:17.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 msgr2=0x7fd8c8071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:17.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 0x7fd8c8071720 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400ebf0 tx=0x7fd8c400b040 comp rx=0 tx=0).stop 2023-06-22T18:09:17.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 shutdown_connections 2023-06-22T18:09:17.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd8b409b700 0x7fd8b409dbb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:17.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c80fe7a0 0x7fd8c80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:17.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80fcc00 0x7fd8c8071720 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:17.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 --2- 172.21.15.39:0/197561060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8c80fde10 0x7fd8c806fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:17.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 >> 172.21.15.39:0/197561060 conn(0x7fd8c80f8320 msgr2=0x7fd8c80fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:09:17.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 shutdown_connections 2023-06-22T18:09:17.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:09:17.067+0000 7fd8cfec9700 1 -- 172.21.15.39:0/197561060 wait complete. 2023-06-22T18:09:17.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:16 smithi101 ceph-mon[116267]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:17.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:16 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:16 smithi133 ceph-mon[116867]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:17.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:16 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.619 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.9.service 2023-06-22T18:09:17.623 INFO:tasks.cephadm:Deploying osd.10 on smithi101 with /dev/vg_nvme/lv_1... 2023-06-22T18:09:17.623 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-22T18:09:17.693 INFO:journalctl@ceph.osd.9.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:09:17.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:17 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:17.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:17 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:17 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:17.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:17 smithi039 ceph-mon[109976]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-22T18:09:18.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:17 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:18.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:17 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:18.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:17 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:18.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:17 smithi101 ceph-mon[116267]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-22T18:09:18.214 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:09:18.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:17 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:18.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:17 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:18.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:17 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:18.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:17 smithi133 ceph-mon[116867]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-22T18:09:19.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:19 smithi039 ceph-mon[109976]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:19.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:19 smithi039 ceph-mon[109976]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-22T18:09:19.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:19 smithi039 ceph-mon[109976]: osdmap e65: 10 total, 9 up, 10 in 2023-06-22T18:09:19.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:19 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:19.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:19 smithi039 ceph-mon[109976]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:09:19.196 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:09:19 smithi039 conmon[147329]: 2023-06-22T18:09:19.068+0000 7fbc8c49f700 -1 osd.9 0 waiting for initial osdmap 2023-06-22T18:09:19.196 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:09:19 smithi039 conmon[147329]: 2023-06-22T18:09:19.084+0000 7fbc8729a700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:09:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:19 smithi101 ceph-mon[116267]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:19 smithi101 ceph-mon[116267]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-22T18:09:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:19 smithi101 ceph-mon[116267]: osdmap e65: 10 total, 9 up, 10 in 2023-06-22T18:09:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:19 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:19.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:19 smithi101 ceph-mon[116267]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:09:19.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:19 smithi133 ceph-mon[116867]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:19.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:19 smithi133 ceph-mon[116867]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-22T18:09:19.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:19 smithi133 ceph-mon[116867]: osdmap e65: 10 total, 9 up, 10 in 2023-06-22T18:09:19.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:19 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:19.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:19 smithi133 ceph-mon[116867]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2023-06-22T18:09:20.002 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-22T18:09:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-22T18:09:20.003 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-06-22T18:09:20.003 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-06-22T18:09:20.003 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113413 s, 925 MB/s 2023-06-22T18:09:20.004 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-06-22T18:09:20.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:20 smithi101 ceph-mon[116267]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:09:20.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:20 smithi101 ceph-mon[116267]: osdmap e66: 10 total, 9 up, 10 in 2023-06-22T18:09:20.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:20.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:20.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:20 smithi039 ceph-mon[109976]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:09:20.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:20 smithi039 ceph-mon[109976]: osdmap e66: 10 total, 9 up, 10 in 2023-06-22T18:09:20.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:20.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:20.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:20 smithi133 ceph-mon[116867]: from='osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2023-06-22T18:09:20.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:20 smithi133 ceph-mon[116867]: osdmap e66: 10 total, 9 up, 10 in 2023-06-22T18:09:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:20.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:20.748 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_1 2023-06-22T18:09:21.327 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-22T18:09:21.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:09:21.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:09:21.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:21.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:21.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983] boot 2023-06-22T18:09:21.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: osdmap e67: 10 total, 10 up, 10 in 2023-06-22T18:09:21.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:21.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:21.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:21.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:21 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:09:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:09:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983] boot 2023-06-22T18:09:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: osdmap e67: 10 total, 10 up, 10 in 2023-06-22T18:09:21.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:21.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:21.447 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:21 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:09:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:09:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-06-22T18:09:21.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: osd.9 [v2:172.21.15.39:6826/881304983,v1:172.21.15.39:6827/881304983] boot 2023-06-22T18:09:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: osdmap e67: 10 total, 10 up, 10 in 2023-06-22T18:09:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-22T18:09:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:21.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:21.487 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:21.487 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:21.487 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:21 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:22.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:22 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi039 2023-06-22T18:09:22.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:22 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi039 to 4316M 2023-06-22T18:09:22.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:22 smithi101 ceph-mon[116267]: osdmap e68: 10 total, 10 up, 10 in 2023-06-22T18:09:22.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:22 smithi101 ceph-mon[116267]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:22.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:22 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi039 2023-06-22T18:09:22.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:22 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi039 to 4316M 2023-06-22T18:09:22.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:22 smithi039 ceph-mon[109976]: osdmap e68: 10 total, 10 up, 10 in 2023-06-22T18:09:22.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:22 smithi039 ceph-mon[109976]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:22 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi039 2023-06-22T18:09:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:22 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi039 to 4316M 2023-06-22T18:09:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:22 smithi133 ceph-mon[116867]: osdmap e68: 10 total, 10 up, 10 in 2023-06-22T18:09:22.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:22 smithi133 ceph-mon[116867]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:22.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 -- 172.21.15.101:0/868903725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 msgr2=0x7fc8180ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 --2- 172.21.15.101:0/868903725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc8180ff7e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc8080049c0 tx=0x7fc808009e30 comp rx=0 tx=0).stop 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 -- 172.21.15.101:0/868903725 shutdown_connections 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 --2- 172.21.15.101:0/868903725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc818102880 0x7fc818104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 --2- 172.21.15.101:0/868903725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc8180ff7e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 --2- 172.21.15.101:0/868903725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8180ffdb0 0x7fc8181021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 -- 172.21.15.101:0/868903725 >> 172.21.15.101:0/868903725 conn(0x7fc8180f7230 msgr2=0x7fc8180f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:09:22.887 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 -- 172.21.15.101:0/868903725 shutdown_connections 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.882+0000 7fc820248700 1 -- 172.21.15.101:0/868903725 wait complete. 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 Processor -- start 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 -- start start 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc81810dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8180ffdb0 0x7fc81810e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc818102880 0x7fc818115090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181156a0 con 0x7fc818102880 2023-06-22T18:09:22.888 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818115810 con 0x7fc8180669c0 2023-06-22T18:09:22.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc820248700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181159d0 con 0x7fc8180ffdb0 2023-06-22T18:09:22.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc81810dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:22.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc81810dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:44344/0 (socket says 172.21.15.101:44344) 2023-06-22T18:09:22.889 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 -- 172.21.15.101:0/1136096864 learned_addr learned my addr 172.21.15.101:0/1136096864 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81d7e3700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8180ffdb0 0x7fc81810e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 -- 172.21.15.101:0/1136096864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8180ffdb0 msgr2=0x7fc81810e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8180ffdb0 0x7fc81810e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 -- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc818102880 msgr2=0x7fc818115090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc818102880 0x7fc818115090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 -- 172.21.15.101:0/1136096864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808005040 con 0x7fc8180669c0 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.883+0000 7fc81dfe4700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc81810dac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc80800bfa0 tx=0x7fc808009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:09:22.890 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.884+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800e040 con 0x7fc8180669c0 2023-06-22T18:09:22.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.884+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8080097f0 con 0x7fc8180669c0 2023-06-22T18:09:22.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.884+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80801c3f0 con 0x7fc8180669c0 2023-06-22T18:09:22.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.884+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818115c50 con 0x7fc8180669c0 2023-06-22T18:09:22.891 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.884+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818116060 con 0x7fc8180669c0 2023-06-22T18:09:22.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.885+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc818108520 con 0x7fc8180669c0 2023-06-22T18:09:22.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.885+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fc8080135b0 con 0x7fc8180669c0 2023-06-22T18:09:22.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.886+0000 7fc8167fc700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc80009b680 0x7fc80009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:22.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.886+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fc8080c58b0 con 0x7fc8180669c0 2023-06-22T18:09:22.892 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.886+0000 7fc81d7e3700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc80009b680 0x7fc80009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:22.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.887+0000 7fc81d7e3700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc80009b680 0x7fc80009db30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc80c006fd0 tx=0x7fc80c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:09:22.893 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:22.890+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fc808091910 con 0x7fc8180669c0 2023-06-22T18:09:23.083 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:23.081+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8180fef70 con 0x7fc80009b680 2023-06-22T18:09:23.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:23 smithi039 ceph-mon[109976]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:09:23.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:23 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:09:23.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:23 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:09:23.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:23 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:23.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:23 smithi133 ceph-mon[116867]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:09:23.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:23 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:09:23.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:23 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:09:23.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:23 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:23.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:23 smithi101 ceph-mon[116267]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:09:23.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:23 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:09:23.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:23 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:09:23.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:23 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:24.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:24 smithi039 ceph-mon[109976]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:24.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:24 smithi133 ceph-mon[116867]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:24.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:24 smithi101 ceph-mon[116267]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/1481467744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]: dispatch 2023-06-22T18:09:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]: dispatch 2023-06-22T18:09:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]': finished 2023-06-22T18:09:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: osdmap e69: 11 total, 10 up, 11 in 2023-06-22T18:09:27.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:27.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:26 smithi101 ceph-mon[116267]: from='client.? 172.21.15.101:0/1170205506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:27.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:27.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/1481467744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]: dispatch 2023-06-22T18:09:27.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]: dispatch 2023-06-22T18:09:27.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]': finished 2023-06-22T18:09:27.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: osdmap e69: 11 total, 10 up, 11 in 2023-06-22T18:09:27.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:27.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:26 smithi039 ceph-mon[109976]: from='client.? 172.21.15.101:0/1170205506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/1481467744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]: dispatch 2023-06-22T18:09:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]: dispatch 2023-06-22T18:09:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4048ac5c-ffb1-4b04-85ec-6afc2add5b4a"}]': finished 2023-06-22T18:09:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: osdmap e69: 11 total, 10 up, 11 in 2023-06-22T18:09:27.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:27.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:26 smithi133 ceph-mon[116867]: from='client.? 172.21.15.101:0/1170205506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:29.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:28 smithi101 ceph-mon[116267]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:29.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:28 smithi039 ceph-mon[109976]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:29.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:28 smithi133 ceph-mon[116867]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:31.054 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:30 smithi101 ceph-mon[116267]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:31.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:30 smithi039 ceph-mon[109976]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:31.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:30 smithi133 ceph-mon[116867]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:32.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:32 smithi101 ceph-mon[116267]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:33.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:32 smithi039 ceph-mon[109976]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:33.233 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:32 smithi133 ceph-mon[116867]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:34.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:33 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-22T18:09:34.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:33 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:34.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:33 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-22T18:09:34.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:33 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:34.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:33 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-22T18:09:34.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:33 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:35.099 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:34 smithi101 ceph-mon[116267]: Deploying daemon osd.10 on smithi101 2023-06-22T18:09:35.099 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:34 smithi101 ceph-mon[116267]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:35.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:34 smithi039 ceph-mon[109976]: Deploying daemon osd.10 on smithi101 2023-06-22T18:09:35.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:34 smithi039 ceph-mon[109976]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:34 smithi133 ceph-mon[116867]: Deploying daemon osd.10 on smithi101 2023-06-22T18:09:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:34 smithi133 ceph-mon[116867]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:37.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:36 smithi101 ceph-mon[116267]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:37.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:36 smithi039 ceph-mon[109976]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:37.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:36 smithi133 ceph-mon[116867]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:38.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:38 smithi101 ceph-mon[116267]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:39.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:38 smithi039 ceph-mon[109976]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:39.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:38 smithi133 ceph-mon[116867]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:40.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:39 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:40.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:39 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:40.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:39 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:40.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:39 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:40.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:39 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:40.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:39 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:40.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:39 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:40.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:39 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:40.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:39 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:41.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:40 smithi101 ceph-mon[116267]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:41.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:40 smithi039 ceph-mon[109976]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:41.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:40 smithi133 ceph-mon[116867]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:42.609 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.606+0000 7fc8167fc700 1 -- 172.21.15.101:0/1136096864 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc8180fef70 con 0x7fc80009b680 2023-06-22T18:09:42.609 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 10 on host 'smithi101' 2023-06-22T18:09:42.611 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc80009b680 msgr2=0x7fc80009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc80009b680 0x7fc80009db30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc80c006fd0 tx=0x7fc80c008040 comp rx=0 tx=0).stop 2023-06-22T18:09:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 msgr2=0x7fc81810dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc81810dac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc80800bfa0 tx=0x7fc808009e30 comp rx=0 tx=0).stop 2023-06-22T18:09:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 shutdown_connections 2023-06-22T18:09:42.612 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fc80009b680 0x7fc80009db30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc818102880 0x7fc818115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180669c0 0x7fc81810dac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 --2- 172.21.15.101:0/1136096864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8180ffdb0 0x7fc81810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 >> 172.21.15.101:0/1136096864 conn(0x7fc8180f7230 msgr2=0x7fc8180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:09:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 shutdown_connections 2023-06-22T18:09:42.613 INFO:teuthology.orchestra.run.smithi101.stderr:2023-06-22T18:09:42.608+0000 7fc820248700 1 -- 172.21.15.101:0/1136096864 wait complete. 2023-06-22T18:09:43.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-22T18:09:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:42 smithi101 ceph-mon[116267]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-22T18:09:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-22T18:09:43.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:42 smithi039 ceph-mon[109976]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-22T18:09:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:09:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-22T18:09:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:42 smithi133 ceph-mon[116867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-22T18:09:43.281 DEBUG:teuthology.orchestra.run.smithi101:osd.10> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.10.service 2023-06-22T18:09:43.283 INFO:tasks.cephadm:Deploying osd.11 on smithi133 with /dev/vg_nvme/lv_1... 2023-06-22T18:09:43.284 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-22T18:09:43.309 INFO:journalctl@ceph.osd.10.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:09:43.874 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:09:44.807 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:09:44 smithi101 conmon[143689]: 2023-06-22T18:09:44.584+0000 7fca1bbee700 -1 osd.10 0 waiting for initial osdmap 2023-06-22T18:09:44.807 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:09:44 smithi101 conmon[143689]: 2023-06-22T18:09:44.599+0000 7fca189ed700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:09:44.807 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:44 smithi101 ceph-mon[116267]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-22T18:09:44.808 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:44 smithi101 ceph-mon[116267]: osdmap e70: 11 total, 10 up, 11 in 2023-06-22T18:09:44.808 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:44 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:44.808 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:44 smithi101 ceph-mon[116267]: from='osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:09:44.808 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:44 smithi101 ceph-mon[116267]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:09:44.808 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:44 smithi101 ceph-mon[116267]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:44.907 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:44 smithi133 ceph-mon[116867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-22T18:09:44.908 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:44 smithi133 ceph-mon[116867]: osdmap e70: 11 total, 10 up, 11 in 2023-06-22T18:09:44.908 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:44 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:44.908 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:44 smithi133 ceph-mon[116867]: from='osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:09:44.908 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:44 smithi133 ceph-mon[116867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:09:44.908 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:44 smithi133 ceph-mon[116867]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:44.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:44 smithi039 ceph-mon[109976]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-22T18:09:44.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:44 smithi039 ceph-mon[109976]: osdmap e70: 11 total, 10 up, 11 in 2023-06-22T18:09:44.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:44 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:44.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:44 smithi039 ceph-mon[109976]: from='osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:09:44.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:44 smithi039 ceph-mon[109976]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-06-22T18:09:44.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:44 smithi039 ceph-mon[109976]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-06-22T18:09:45.655 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-22T18:09:45.656 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-22T18:09:45.656 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-06-22T18:09:45.656 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-06-22T18:09:45.656 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112321 s, 934 MB/s 2023-06-22T18:09:45.656 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-06-22T18:09:45.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:45 smithi101 ceph-mon[116267]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:09:45.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:45 smithi101 ceph-mon[116267]: osdmap e71: 11 total, 10 up, 11 in 2023-06-22T18:09:45.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:45.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:45.881 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:45 smithi133 ceph-mon[116867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:09:45.881 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:45 smithi133 ceph-mon[116867]: osdmap e71: 11 total, 10 up, 11 in 2023-06-22T18:09:45.881 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:45.881 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:45.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:45 smithi039 ceph-mon[109976]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-06-22T18:09:45.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:45 smithi039 ceph-mon[109976]: osdmap e71: 11 total, 10 up, 11 in 2023-06-22T18:09:45.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:45.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:46.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2023-06-22T18:09:46.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:09:46.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:09:46.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778] boot 2023-06-22T18:09:46.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: osdmap e72: 11 total, 11 up, 11 in 2023-06-22T18:09:46.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:46.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:46.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.856 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.856 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.856 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:46.857 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:46.857 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:46 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:09:46.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:09:46.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778] boot 2023-06-22T18:09:46.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: osdmap e72: 11 total, 11 up, 11 in 2023-06-22T18:09:46.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:46.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:46.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.947 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:46.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:46.948 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:46 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:09:46.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:09:46.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: osd.10 [v2:172.21.15.101:6824/1823054778,v1:172.21.15.101:6825/1823054778] boot 2023-06-22T18:09:46.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: osdmap e72: 11 total, 11 up, 11 in 2023-06-22T18:09:46.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-22T18:09:46.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:46.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:09:46.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:46.988 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:09:46.988 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:46 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:09:46.989 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:09:47.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:47 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi101 2023-06-22T18:09:47.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:47 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi101 to 4293M 2023-06-22T18:09:47.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:47 smithi039 ceph-mon[109976]: osdmap e73: 11 total, 11 up, 11 in 2023-06-22T18:09:47.947 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:47 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi101 2023-06-22T18:09:47.947 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:47 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi101 to 4293M 2023-06-22T18:09:47.947 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:47 smithi133 ceph-mon[116867]: osdmap e73: 11 total, 11 up, 11 in 2023-06-22T18:09:48.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:47 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi101 2023-06-22T18:09:48.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:47 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi101 to 4293M 2023-06-22T18:09:48.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:47 smithi101 ceph-mon[116267]: osdmap e73: 11 total, 11 up, 11 in 2023-06-22T18:09:48.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 -- 172.21.15.133:0/3819954596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 msgr2=0x7f1410102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:48.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 --2- 172.21.15.133:0/3819954596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f1410102450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13f80049c0 tx=0x7f13f8009e30 comp rx=0 tx=0).stop 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 -- 172.21.15.133:0/3819954596 shutdown_connections 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 --2- 172.21.15.133:0/3819954596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1410102990 0x7f1410104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 --2- 172.21.15.133:0/3819954596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14100fd750 0x7f14100ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 --2- 172.21.15.133:0/3819954596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f1410102450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 -- 172.21.15.133:0/3819954596 >> 172.21.15.133:0/3819954596 conn(0x7f14100f7230 msgr2=0x7f14100f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 -- 172.21.15.133:0/3819954596 shutdown_connections 2023-06-22T18:09:48.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 -- 172.21.15.133:0/3819954596 wait complete. 2023-06-22T18:09:48.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.406+0000 7f141621e700 1 Processor -- start 2023-06-22T18:09:48.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 -- start start 2023-06-22T18:09:48.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14100fd750 0x7f141010daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:48.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f141010e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:48.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1410102990 0x7f14101150c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14101156d0 con 0x7f1410102990 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1410115840 con 0x7f14100fd750 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1410115a00 con 0x7f1410100070 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f141010e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f141010e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47846/0 (socket says 172.21.15.133:47846) 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 -- 172.21.15.133:0/507096939 learned_addr learned my addr 172.21.15.133:0/507096939 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:09:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f140f7fe700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14100fd750 0x7f141010daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f140ffff700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1410102990 0x7f14101150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 -- 172.21.15.133:0/507096939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14100fd750 msgr2=0x7f141010daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14100fd750 0x7f141010daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 -- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1410102990 msgr2=0x7f14101150c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1410102990 0x7f14101150c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 -- 172.21.15.133:0/507096939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8005040 con 0x7f1410100070 2023-06-22T18:09:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f1406ffd700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f141010e030 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f13f800bfa0 tx=0x7f13f8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:09:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f800e040 con 0x7f1410100070 2023-06-22T18:09:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13f80097f0 con 0x7f1410100070 2023-06-22T18:09:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1410115c80 con 0x7f1410100070 2023-06-22T18:09:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.407+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f801c3f0 con 0x7f1410100070 2023-06-22T18:09:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.408+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14101160e0 con 0x7f1410100070 2023-06-22T18:09:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.409+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14101084f0 con 0x7f1410100070 2023-06-22T18:09:48.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.410+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f13f80135b0 con 0x7f1410100070 2023-06-22T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.410+0000 7f140cff9700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f13fc09b680 0x7f13fc09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.410+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f13f80c64b0 con 0x7f1410100070 2023-06-22T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.411+0000 7f140f7fe700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f13fc09b680 0x7f13fc09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.411+0000 7f140f7fe700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f13fc09b680 0x7f13fc09db30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1400006fd0 tx=0x7f1400008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:09:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.414+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f13f8092350 con 0x7f1410100070 2023-06-22T18:09:48.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:09:48.602+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f14100ff2f0 con 0x7f13fc09b680 2023-06-22T18:09:48.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:48 smithi039 ceph-mon[109976]: osdmap e74: 11 total, 11 up, 11 in 2023-06-22T18:09:48.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:48 smithi039 ceph-mon[109976]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:48.983 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:48 smithi133 ceph-mon[116867]: osdmap e74: 11 total, 11 up, 11 in 2023-06-22T18:09:48.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:48 smithi133 ceph-mon[116867]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:49.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:48 smithi101 ceph-mon[116267]: osdmap e74: 11 total, 11 up, 11 in 2023-06-22T18:09:49.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:48 smithi101 ceph-mon[116267]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:49.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:49 smithi133 ceph-mon[116867]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:09:49.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:09:49.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:49 smithi133 ceph-mon[116867]: osdmap e75: 11 total, 11 up, 11 in 2023-06-22T18:09:49.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:09:49.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:49 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:49.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:49 smithi039 ceph-mon[109976]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:09:49.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:09:49.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:49 smithi039 ceph-mon[109976]: osdmap e75: 11 total, 11 up, 11 in 2023-06-22T18:09:49.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:09:49.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:49 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:50.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:49 smithi101 ceph-mon[116267]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:09:50.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:49 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-22T18:09:50.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:49 smithi101 ceph-mon[116267]: osdmap e75: 11 total, 11 up, 11 in 2023-06-22T18:09:50.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:49 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-22T18:09:50.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:49 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:09:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:50 smithi039 ceph-mon[109976]: osdmap e76: 11 total, 11 up, 11 in 2023-06-22T18:09:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:50 smithi039 ceph-mon[109976]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-22T18:09:50.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:50 smithi133 ceph-mon[116867]: osdmap e76: 11 total, 11 up, 11 in 2023-06-22T18:09:50.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:50 smithi133 ceph-mon[116867]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-22T18:09:51.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:50 smithi101 ceph-mon[116267]: osdmap e76: 11 total, 11 up, 11 in 2023-06-22T18:09:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:50 smithi101 ceph-mon[116267]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-22T18:09:51.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:51 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/3008938115' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]: dispatch 2023-06-22T18:09:51.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:51 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]: dispatch 2023-06-22T18:09:51.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:51 smithi039 ceph-mon[109976]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]': finished 2023-06-22T18:09:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:51 smithi039 ceph-mon[109976]: osdmap e77: 12 total, 11 up, 12 in 2023-06-22T18:09:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:09:51.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:51 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/3008938115' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]: dispatch 2023-06-22T18:09:51.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:51 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]: dispatch 2023-06-22T18:09:51.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:51 smithi133 ceph-mon[116867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]': finished 2023-06-22T18:09:51.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:51 smithi133 ceph-mon[116867]: osdmap e77: 12 total, 11 up, 12 in 2023-06-22T18:09:51.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:09:52.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:51 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/3008938115' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]: dispatch 2023-06-22T18:09:52.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:51 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]: dispatch 2023-06-22T18:09:52.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:51 smithi101 ceph-mon[116267]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "334df2ca-0d0e-4450-bd11-cd8651c44e3e"}]': finished 2023-06-22T18:09:52.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:51 smithi101 ceph-mon[116267]: osdmap e77: 12 total, 11 up, 12 in 2023-06-22T18:09:52.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:09:52.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:52 smithi039 ceph-mon[109976]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-06-22T18:09:52.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:52 smithi039 ceph-mon[109976]: from='client.? 172.21.15.133:0/1611612118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:52.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:52 smithi133 ceph-mon[116867]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-06-22T18:09:52.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:52 smithi133 ceph-mon[116867]: from='client.? 172.21.15.133:0/1611612118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:53.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:52 smithi101 ceph-mon[116267]: pgmap v246: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-06-22T18:09:53.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:52 smithi101 ceph-mon[116267]: from='client.? 172.21.15.133:0/1611612118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-22T18:09:55.010 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:54 smithi133 ceph-mon[116867]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-22T18:09:55.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:54 smithi101 ceph-mon[116267]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-22T18:09:55.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:54 smithi039 ceph-mon[109976]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-22T18:09:56.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:56 smithi133 ceph-mon[116867]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-06-22T18:09:57.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:56 smithi101 ceph-mon[116267]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-06-22T18:09:57.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:56 smithi039 ceph-mon[109976]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-06-22T18:09:58.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:58 smithi133 ceph-mon[116867]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:59.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:58 smithi101 ceph-mon[116267]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:09:59.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:58 smithi039 ceph-mon[109976]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:00.027 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:59 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-22T18:10:00.027 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:09:59 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:00.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:59 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-22T18:10:00.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:09:59 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:59 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-22T18:10:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:09:59 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:00.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:00 smithi133 ceph-mon[116867]: Deploying daemon osd.11 on smithi133 2023-06-22T18:10:00.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:00 smithi133 ceph-mon[116867]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:00.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:00 smithi133 ceph-mon[116867]: overall HEALTH_OK 2023-06-22T18:10:01.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:00 smithi101 ceph-mon[116267]: Deploying daemon osd.11 on smithi133 2023-06-22T18:10:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:00 smithi101 ceph-mon[116267]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:01.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:00 smithi101 ceph-mon[116267]: overall HEALTH_OK 2023-06-22T18:10:01.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:00 smithi039 ceph-mon[109976]: Deploying daemon osd.11 on smithi133 2023-06-22T18:10:01.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:00 smithi039 ceph-mon[109976]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:01.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:00 smithi039 ceph-mon[109976]: overall HEALTH_OK 2023-06-22T18:10:03.073 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:02 smithi133 ceph-mon[116867]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:03.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:02 smithi101 ceph-mon[116267]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:03.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:02 smithi039 ceph-mon[109976]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:05.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:04 smithi101 ceph-mon[116267]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:05.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:04 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:04 smithi039 ceph-mon[109976]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:05.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:04 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:05.233 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:04 smithi133 ceph-mon[116867]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:05.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:04 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:06.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:06 smithi133 ceph-mon[116867]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:07.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:06 smithi101 ceph-mon[116267]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:07.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:06 smithi039 ceph-mon[109976]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:07.784 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 11 on host 'smithi133' 2023-06-22T18:10:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.779+0000 7f140cff9700 1 -- 172.21.15.133:0/507096939 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f14100ff2f0 con 0x7f13fc09b680 2023-06-22T18:10:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f13fc09b680 msgr2=0x7f13fc09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f13fc09b680 0x7f13fc09db30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1400006fd0 tx=0x7f1400008040 comp rx=0 tx=0).stop 2023-06-22T18:10:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 msgr2=0x7f141010e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f141010e030 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f13f800bfa0 tx=0x7f13f8009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 shutdown_connections 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f13fc09b680 0x7f13fc09db30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1410102990 0x7f14101150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14100fd750 0x7f141010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 --2- 172.21.15.133:0/507096939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1410100070 0x7f141010e030 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 >> 172.21.15.133:0/507096939 conn(0x7f14100f7230 msgr2=0x7f14100febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 shutdown_connections 2023-06-22T18:10:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:07.782+0000 7f141621e700 1 -- 172.21.15.133:0/507096939 wait complete. 2023-06-22T18:10:08.439 DEBUG:teuthology.orchestra.run.smithi133:osd.11> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.11.service 2023-06-22T18:10:08.442 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-06-22T18:10:08.443 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-22T18:10:08.471 INFO:journalctl@ceph.osd.11.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:10:08.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:08.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:08.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:08.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:09.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: from='osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-22T18:10:09.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:08 smithi101 ceph-mon[116267]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-22T18:10:09.143 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:09.143 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.143 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.143 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.143 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.144 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: from='osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-22T18:10:09.144 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:08 smithi133 ceph-mon[116867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-22T18:10:09.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:09.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: from='osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-22T18:10:09.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:08 smithi039 ceph-mon[109976]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-22T18:10:09.223 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:10:10.175 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:09 smithi039 ceph-mon[109976]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-22T18:10:10.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:09 smithi039 ceph-mon[109976]: osdmap e78: 12 total, 11 up, 12 in 2023-06-22T18:10:10.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:09 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:10.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:09 smithi039 ceph-mon[109976]: from='osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:10:10.176 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:09 smithi039 ceph-mon[109976]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:10:10.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:09 smithi133 ceph-mon[116867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-22T18:10:10.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:09 smithi133 ceph-mon[116867]: osdmap e78: 12 total, 11 up, 12 in 2023-06-22T18:10:10.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:09 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:10.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:09 smithi133 ceph-mon[116867]: from='osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:10:10.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:09 smithi133 ceph-mon[116867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:10:10.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:10:09 smithi133 conmon[145608]: 2023-06-22T18:10:09.875+0000 7efc9f68e700 -1 osd.11 0 waiting for initial osdmap 2023-06-22T18:10:10.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:10:09 smithi133 conmon[145608]: 2023-06-22T18:10:09.890+0000 7efc98485700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-22T18:10:10.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:09 smithi101 ceph-mon[116267]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-22T18:10:10.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:09 smithi101 ceph-mon[116267]: osdmap e78: 12 total, 11 up, 12 in 2023-06-22T18:10:10.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:09 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:10.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:09 smithi101 ceph-mon[116267]: from='osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:10:10.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:09 smithi101 ceph-mon[116267]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-06-22T18:10:10.743 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 -- 172.21.15.39:0/1239565776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fde90 msgr2=0x7f3e600fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 --2- 172.21.15.39:0/1239565776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fde90 0x7f3e600fe2e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e50004900 tx=0x7f3e50009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 -- 172.21.15.39:0/1239565776 shutdown_connections 2023-06-22T18:10:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 --2- 172.21.15.39:0/1239565776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fe820 0x7f3e601050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 --2- 172.21.15.39:0/1239565776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e600fcc80 0x7f3e600fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 --2- 172.21.15.39:0/1239565776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fde90 0x7f3e600fe2e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 -- 172.21.15.39:0/1239565776 >> 172.21.15.39:0/1239565776 conn(0x7f3e600f83a0 msgr2=0x7f3e600fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 -- 172.21.15.39:0/1239565776 shutdown_connections 2023-06-22T18:10:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 -- 172.21.15.39:0/1239565776 wait complete. 2023-06-22T18:10:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.741+0000 7f3e67a29700 1 Processor -- start 2023-06-22T18:10:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 -- start start 2023-06-22T18:10:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 0x7f3e6010fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fde90 0x7f3e60110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e600fe820 0x7f3e601170a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601176b0 con 0x7f3e600fcc80 2023-06-22T18:10:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60117820 con 0x7f3e600fe820 2023-06-22T18:10:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e67a29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601179e0 con 0x7f3e600fde90 2023-06-22T18:10:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e657c5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 0x7f3e6010fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e657c5700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 0x7f3e6010fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41434/0 (socket says 172.21.15.39:41434) 2023-06-22T18:10:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e64fc4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fde90 0x7f3e60110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e657c5700 1 -- 172.21.15.39:0/4247597835 learned_addr learned my addr 172.21.15.39:0/4247597835 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.742+0000 7f3e65fc6700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e600fe820 0x7f3e601170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e657c5700 1 -- 172.21.15.39:0/4247597835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fde90 msgr2=0x7f3e60110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e657c5700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fde90 0x7f3e60110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e657c5700 1 -- 172.21.15.39:0/4247597835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e600fe820 msgr2=0x7f3e601170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e657c5700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e600fe820 0x7f3e601170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e657c5700 1 -- 172.21.15.39:0/4247597835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50005040 con 0x7f3e600fcc80 2023-06-22T18:10:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e64fc4700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fde90 0x7f3e60110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e657c5700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 0x7f3e6010fad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00ebf0 tx=0x7f3e5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0047d0 con 0x7f3e600fcc80 2023-06-22T18:10:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e60117c60 con 0x7f3e600fcc80 2023-06-22T18:10:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5c00f040 con 0x7f3e600fcc80 2023-06-22T18:10:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00c980 con 0x7f3e600fcc80 2023-06-22T18:10:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.743+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e601181a0 con 0x7f3e600fcc80 2023-06-22T18:10:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.744+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6010a420 con 0x7f3e600fcc80 2023-06-22T18:10:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.746+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f3e5c0090d0 con 0x7f3e600fcc80 2023-06-22T18:10:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.747+0000 7f3e55ffb700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e4c09b680 0x7f3e4c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.748+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6843+0+0 (secure 0 0 0) 0x7f3e5c0c4320 con 0x7f3e600fcc80 2023-06-22T18:10:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.748+0000 7f3e64fc4700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e4c09b680 0x7f3e4c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.749+0000 7f3e64fc4700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e4c09b680 0x7f3e4c09db30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009f80 tx=0x7f3e50009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.752+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f3e5c04da00 con 0x7f3e600fcc80 2023-06-22T18:10:10.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.958+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f3e60108400 con 0x7f3e600fcc80 2023-06-22T18:10:10.961 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.958+0000 7f3e55ffb700 1 -- 172.21.15.39:0/4247597835 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f3e5c0900a0 con 0x7f3e600fcc80 2023-06-22T18:10:10.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e4c09b680 msgr2=0x7f3e4c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e4c09b680 0x7f3e4c09db30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009f80 tx=0x7f3e50009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 msgr2=0x7f3e6010fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 0x7f3e6010fad0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00ebf0 tx=0x7f3e5c00b040 comp rx=0 tx=0).stop 2023-06-22T18:10:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 shutdown_connections 2023-06-22T18:10:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f3e4c09b680 0x7f3e4c09db30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e600fe820 0x7f3e601170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e600fcc80 0x7f3e6010fad0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 --2- 172.21.15.39:0/4247597835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e600fde90 0x7f3e60110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:10.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 >> 172.21.15.39:0/4247597835 conn(0x7f3e600f83a0 msgr2=0x7f3e60103790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:10.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 shutdown_connections 2023-06-22T18:10:10.965 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:10.961+0000 7f3e67a29700 1 -- 172.21.15.39:0/4247597835 wait complete. 2023-06-22T18:10:11.168 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:10 smithi039 ceph-mon[109976]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:11.168 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:10 smithi039 ceph-mon[109976]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:10:11.168 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:10 smithi039 ceph-mon[109976]: osdmap e79: 12 total, 11 up, 12 in 2023-06-22T18:10:11.168 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:11.169 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:10 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:11.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:10 smithi133 ceph-mon[116867]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:11.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:10 smithi133 ceph-mon[116867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:10:11.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:10 smithi133 ceph-mon[116867]: osdmap e79: 12 total, 11 up, 12 in 2023-06-22T18:10:11.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:10 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:11.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:10 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:11.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:10 smithi101 ceph-mon[116267]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:11.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:10 smithi101 ceph-mon[116267]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-06-22T18:10:11.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:10 smithi101 ceph-mon[116267]: osdmap e79: 12 total, 11 up, 12 in 2023-06-22T18:10:11.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:10 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:11.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:10 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:11.489 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1687457410,"num_in_osds":12,"osd_in_since":1687457391,"num_remapped_pgs":0} 2023-06-22T18:10:11.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-22T18:10:12.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: purged_snaps scrub starts 2023-06-22T18:10:12.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: purged_snaps scrub ok 2023-06-22T18:10:12.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943] boot 2023-06-22T18:10:12.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: osdmap e80: 12 total, 12 up, 12 in 2023-06-22T18:10:12.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:12.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/4247597835' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-22T18:10:12.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: Detected new or changed devices on smithi133 2023-06-22T18:10:12.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: Adjusting osd_memory_target on smithi133 to 4293M 2023-06-22T18:10:12.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:12.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:11 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: purged_snaps scrub starts 2023-06-22T18:10:12.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: purged_snaps scrub ok 2023-06-22T18:10:12.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943] boot 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: osdmap e80: 12 total, 12 up, 12 in 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/4247597835' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: Detected new or changed devices on smithi133 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: Adjusting osd_memory_target on smithi133 to 4293M 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:12.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:12.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:12.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:12.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:12.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:11 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.259 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:10:12.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: purged_snaps scrub starts 2023-06-22T18:10:12.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: purged_snaps scrub ok 2023-06-22T18:10:12.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: osd.11 [v2:172.21.15.133:6824/93348943,v1:172.21.15.133:6825/93348943] boot 2023-06-22T18:10:12.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: osdmap e80: 12 total, 12 up, 12 in 2023-06-22T18:10:12.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-22T18:10:12.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/4247597835' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: Detected new or changed devices on smithi133 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: Adjusting osd_memory_target on smithi133 to 4293M 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:12.355 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:12.356 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:11 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:13.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:12 smithi039 ceph-mon[109976]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:13.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:12 smithi039 ceph-mon[109976]: osdmap e81: 12 total, 12 up, 12 in 2023-06-22T18:10:13.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:12 smithi133 ceph-mon[116867]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:13.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:12 smithi133 ceph-mon[116867]: osdmap e81: 12 total, 12 up, 12 in 2023-06-22T18:10:13.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:12 smithi101 ceph-mon[116267]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-06-22T18:10:13.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:12 smithi101 ceph-mon[116267]: osdmap e81: 12 total, 12 up, 12 in 2023-06-22T18:10:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 -- 172.21.15.39:0/1671002652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 msgr2=0x7fd9a80fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 --2- 172.21.15.39:0/1671002652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a80fe030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9980049c0 tx=0x7fd998009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 -- 172.21.15.39:0/1671002652 shutdown_connections 2023-06-22T18:10:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 --2- 172.21.15.39:0/1671002652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a8107210 0x7fd9a81095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 --2- 172.21.15.39:0/1671002652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a80fe030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 --2- 172.21.15.39:0/1671002652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9a80fe600 0x7fd9a8106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 -- 172.21.15.39:0/1671002652 >> 172.21.15.39:0/1671002652 conn(0x7fd9a8072c30 msgr2=0x7fd9a8073030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 -- 172.21.15.39:0/1671002652 shutdown_connections 2023-06-22T18:10:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.766+0000 7fd9ad740700 1 -- 172.21.15.39:0/1671002652 wait complete. 2023-06-22T18:10:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.767+0000 7fd9ad740700 1 Processor -- start 2023-06-22T18:10:13.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.767+0000 7fd9ad740700 1 -- start start 2023-06-22T18:10:13.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9ad740700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a8115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:13.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9ad740700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a80fe600 0x7fd9a8115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:13.773 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9ad740700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9a8107210 0x7fd9a810fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9ad740700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8110180 con 0x7fd9a80fdc60 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9ad740700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81102f0 con 0x7fd9a8107210 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9ad740700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81104b0 con 0x7fd9a80fe600 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a8115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a6ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a8115ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41454/0 (socket says 172.21.15.39:41454) 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a6ffd700 1 -- 172.21.15.39:0/3692776496 learned_addr learned my addr 172.21.15.39:0/3692776496 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a77fe700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9a8107210 0x7fd9a810fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a67fc700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a80fe600 0x7fd9a8115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a6ffd700 1 -- 172.21.15.39:0/3692776496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a80fe600 msgr2=0x7fd9a8115ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.768+0000 7fd9a6ffd700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a80fe600 0x7fd9a8115ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd9a6ffd700 1 -- 172.21.15.39:0/3692776496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9a8107210 msgr2=0x7fd9a810fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd9a6ffd700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9a8107210 0x7fd9a810fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd9a6ffd700 1 -- 172.21.15.39:0/3692776496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998005040 con 0x7fd9a80fdc60 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd9a6ffd700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a8115ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd99800b8f0 tx=0x7fd998009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99800e040 con 0x7fd9a80fdc60 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a8110730 con 0x7fd9a80fdc60 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd998009670 con 0x7fd9a80fdc60 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998013cd0 con 0x7fd9a80fdc60 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.769+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a811caa0 con 0x7fd9a80fdc60 2023-06-22T18:10:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.770+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a804c1b0 con 0x7fd9a80fdc60 2023-06-22T18:10:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.772+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fd9980091a0 con 0x7fd9a80fdc60 2023-06-22T18:10:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.773+0000 7fd98f7fe700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd99009b680 0x7fd99009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.774+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fd9980c59f0 con 0x7fd9a80fdc60 2023-06-22T18:10:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.774+0000 7fd9a67fc700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd99009b680 0x7fd99009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.774+0000 7fd9a67fc700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd99009b680 0x7fd99009db30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8111350 tx=0x7fd99c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.777+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fd998091650 con 0x7fd9a80fdc60 2023-06-22T18:10:13.982 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.979+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9a811cc30 con 0x7fd9a80fdc60 2023-06-22T18:10:13.983 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.980+0000 7fd98f7fe700 1 -- 172.21.15.39:0/3692776496 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18120 (secure 0 0 0) 0x7fd998094c70 con 0x7fd9a80fdc60 2023-06-22T18:10:13.983 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:13.984 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","created":"2023-06-22T18:02:53.160326+0000","modified":"2023-06-22T18:10:12.224478+0000","last_up_change":"2023-06-22T18:10:10.873966+0000","last_in_change":"2023-06-22T18:09:51.312474+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T18:06:20.190713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f55d12a1-86b5-42a6-ad35-d5eaf3d99c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6803","nonce":471291770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6805","nonce":471291770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6809","nonce":471291770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6807","nonce":471291770}]},"public_addr":"172.21.15.39:6803/471291770","cluster_addr":"172.21.15.39:6805/471291770","heartbeat_back_addr":"172.21.15.39:6809/471291770","heartbeat_front_addr":"172.21.15.39:6807/471291770","state":["exists","up"]},{"osd":1,"uuid":"4d8fb427-c4b2-45c3-ac42-dea08c42e4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6801","nonce":840111728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6803","nonce":840111728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6807","nonce":840111728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6805","nonce":840111728}]},"public_addr":"172.21.15.101:6801/840111728","cluster_addr":"172.21.15.101:6803/840111728","heartbeat_back_addr":"172.21.15.101:6807/840111728","heartbeat_front_addr":"172.21.15.101:6805/840111728","state":["exists","up"]},{"osd":2,"uuid":"9c714cec-14fc-4aed-b3f1-f0773edd197c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6801","nonce":3165338416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6803","nonce":3165338416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6807","nonce":3165338416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6805","nonce":3165338416}]},"public_addr":"172.21.15.133:6801/3165338416","cluster_addr":"172.21.15.133:6803/3165338416","heartbeat_back_addr":"172.21.15.133:6807/3165338416","heartbeat_front_addr":"172.21.15.133:6805/3165338416","state":["exists","up"]},{"osd":3,"uuid":"71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6811","nonce":3695104438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6813","nonce":3695104438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6817","nonce":3695104438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6815","nonce":3695104438}]},"public_addr":"172.21.15.39:6811/3695104438","cluster_addr":"172.21.15.39:6813/3695104438","heartbeat_back_addr":"172.21.15.39:6817/3695104438","heartbeat_front_addr":"172.21.15.39:6815/3695104438","state":["exists","up"]},{"osd":4,"uuid":"c9ecace5-1032-4f25-a5f3-84ec819c83ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6809","nonce":2726745315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6811","nonce":2726745315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6815","nonce":2726745315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6813","nonce":2726745315}]},"public_addr":"172.21.15.101:6809/2726745315","cluster_addr":"172.21.15.101:6811/2726745315","heartbeat_back_addr":"172.21.15.101:6815/2726745315","heartbeat_front_addr":"172.21.15.101:6813/2726745315","state":["exists","up"]},{"osd":5,"uuid":"379ed499-68d9-4032-bafd-31cb02995388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6809","nonce":1467210441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6811","nonce":1467210441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6815","nonce":1467210441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6813","nonce":1467210441}]},"public_addr":"172.21.15.133:6809/1467210441","cluster_addr":"172.21.15.133:6811/1467210441","heartbeat_back_addr":"172.21.15.133:6815/1467210441","heartbeat_front_addr":"172.21.15.133:6813/1467210441","state":["exists","up"]},{"osd":6,"uuid":"9cf1482c-6b66-4260-a892-da6969cc8574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6819","nonce":4262819120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6821","nonce":4262819120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6825","nonce":4262819120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6823","nonce":4262819120}]},"public_addr":"172.21.15.39:6819/4262819120","cluster_addr":"172.21.15.39:6821/4262819120","heartbeat_back_addr":"172.21.15.39:6825/4262819120","heartbeat_front_addr":"172.21.15.39:6823/4262819120","state":["exists","up"]},{"osd":7,"uuid":"7b535b3c-1d36-44c9-8c22-132e6d31b0ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6817","nonce":3667650401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6819","nonce":3667650401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6823","nonce":3667650401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6821","nonce":3667650401}]},"public_addr":"172.21.15.101:6817/3667650401","cluster_addr":"172.21.15.101:6819/3667650401","heartbeat_back_addr":"172.21.15.101:6823/3667650401","heartbeat_front_addr":"172.21.15.101:6821/3667650401","state":["exists","up"]},{"osd":8,"uuid":"60376342-edba-4eb8-9907-950963e6c3e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6817","nonce":1333554117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6819","nonce":1333554117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6823","nonce":1333554117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6821","nonce":1333554117}]},"public_addr":"172.21.15.133:6817/1333554117","cluster_addr":"172.21.15.133:6819/1333554117","heartbeat_back_addr":"172.21.15.133:6823/1333554117","heartbeat_front_addr":"172.21.15.133:6821/1333554117","state":["exists","up"]},{"osd":9,"uuid":"c09c5c50-6e36-4e8e-b41c-50fb93a2ea92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6827","nonce":881304983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6829","nonce":881304983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6833","nonce":881304983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6831","nonce":881304983}]},"public_addr":"172.21.15.39:6827/881304983","cluster_addr":"172.21.15.39:6829/881304983","heartbeat_back_addr":"172.21.15.39:6833/881304983","heartbeat_front_addr":"172.21.15.39:6831/881304983","state":["exists","up"]},{"osd":10,"uuid":"4048ac5c-ffb1-4b04-85ec-6afc2add5b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6825","nonce":1823054778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6827","nonce":1823054778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6831","nonce":1823054778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6829","nonce":1823054778}]},"public_addr":"172.21.15.101:6825/1823054778","cluster_addr":"172.21.15.101:6827/1823054778","heartbeat_back_addr":"172.21.15.101:6831/1823054778","heartbeat_front_addr":"172.21.15.101:6829/1823054778","state":["exists","up"]},{"osd":11,"uuid":"334df2ca-0d0e-4450-bd11-cd8651c44e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6825","nonce":93348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6827","nonce":93348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6831","nonce":93348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6829","nonce":93348943}]},"public_addr":"172.21.15.133:6825/93348943","cluster_addr":"172.21.15.133:6827/93348943","heartbeat_back_addr":"172.21.15.133:6831/93348943","heartbeat_front_addr":"172.21.15.133:6829/93348943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:27.345253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:52.437673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:17.253223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:44.552567+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:09.454586+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:34.594288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:01.497366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:26.643829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:51.455308+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:18.528459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:43.697768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:10:09.419715+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/709665139":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/453675837":"2023-06-23T18:03:53.598476+0000","172.21.15.39:6801/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6800/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/2777224344":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/3733013426":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/466669289":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6801/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/1167071234":"2023-06-23T18:03:53.598476+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T18:10:13.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.982+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd99009b680 msgr2=0x7fd99009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:13.985 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.982+0000 7fd9ad740700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd99009b680 0x7fd99009db30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8111350 tx=0x7fd99c00b040 comp rx=0 tx=0).stop 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.982+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 msgr2=0x7fd9a8115ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.982+0000 7fd9ad740700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a8115ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd99800b8f0 tx=0x7fd998009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 shutdown_connections 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd99009b680 0x7fd99009db30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9a8107210 0x7fd9a810fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a80fdc60 0x7fd9a8115ab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 --2- 172.21.15.39:0/3692776496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a80fe600 0x7fd9a8115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 >> 172.21.15.39:0/3692776496 conn(0x7fd9a8072c30 msgr2=0x7fd9a81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 shutdown_connections 2023-06-22T18:10:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:13.983+0000 7fd9ad740700 1 -- 172.21.15.39:0/3692776496 wait complete. 2023-06-22T18:10:14.523 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-22T18:06:20.190713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-22T18:10:14.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-06-22T18:10:15.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:14 smithi039 ceph-mon[109976]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:15.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:14 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3692776496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:15.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:14 smithi133 ceph-mon[116867]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:15.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:14 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3692776496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:15.291 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-22T18:10:15.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:14 smithi101 ceph-mon[116267]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:15.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:14 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3692776496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 -- 172.21.15.39:0/3358146066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 msgr2=0x7f83bc070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 --2- 172.21.15.39:0/3358146066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc070030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0049c0 tx=0x7f83ac009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 -- 172.21.15.39:0/3358146066 shutdown_connections 2023-06-22T18:10:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 --2- 172.21.15.39:0/3358146066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83bc1096b0 0x7f83bc10ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 --2- 172.21.15.39:0/3358146066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc070030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 --2- 172.21.15.39:0/3358146066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83bc070570 0x7f83bc109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 -- 172.21.15.39:0/3358146066 >> 172.21.15.39:0/3358146066 conn(0x7f83bc074af0 msgr2=0x7f83bc074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 -- 172.21.15.39:0/3358146066 shutdown_connections 2023-06-22T18:10:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.912+0000 7f83c1606700 1 -- 172.21.15.39:0/3358146066 wait complete. 2023-06-22T18:10:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 Processor -- start 2023-06-22T18:10:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 -- start start 2023-06-22T18:10:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc115a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83bc070570 0x7f83bc115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83bc1096b0 0x7f83bc10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:16.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1101c0 con 0x7f83bc06fc60 2023-06-22T18:10:16.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc110330 con 0x7f83bc1096b0 2023-06-22T18:10:16.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83c1606700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1104f0 con 0x7f83bc070570 2023-06-22T18:10:16.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83baffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc115a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:16.920 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.913+0000 7f83baffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc115a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47814/0 (socket says 172.21.15.39:47814) 2023-06-22T18:10:16.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 -- 172.21.15.39:0/189468544 learned_addr learned my addr 172.21.15.39:0/189468544 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:16.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83ba7fc700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83bc070570 0x7f83bc115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:16.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83bb7fe700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83bc1096b0 0x7f83bc10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:16.921 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 -- 172.21.15.39:0/189468544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83bc070570 msgr2=0x7f83bc115fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83bc070570 0x7f83bc115fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 -- 172.21.15.39:0/189468544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83bc1096b0 msgr2=0x7f83bc10fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83bc1096b0 0x7f83bc10fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 -- 172.21.15.39:0/189468544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac005040 con 0x7f83bc06fc60 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83baffd700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc115a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009b80 tx=0x7f83ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00e040 con 0x7f83bc06fc60 2023-06-22T18:10:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ac0096f0 con 0x7f83bc06fc60 2023-06-22T18:10:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc110770 con 0x7f83bc06fc60 2023-06-22T18:10:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.914+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac013da0 con 0x7f83bc06fc60 2023-06-22T18:10:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.915+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc11ca40 con 0x7f83bc06fc60 2023-06-22T18:10:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.916+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc04c1b0 con 0x7f83bc06fc60 2023-06-22T18:10:16.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.918+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f83ac009220 con 0x7f83bc06fc60 2023-06-22T18:10:16.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.919+0000 7f83a37fe700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f83a409b680 0x7f83a409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:16.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.919+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f83ac0c6540 con 0x7f83bc06fc60 2023-06-22T18:10:16.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.919+0000 7f83ba7fc700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f83a409b680 0x7f83a409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:16.924 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.920+0000 7f83ba7fc700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f83a409b680 0x7f83a409db30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f83bc1112b0 tx=0x7f83b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:16.925 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:16.921+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f83ac0921a0 con 0x7f83bc06fc60 2023-06-22T18:10:17.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.122+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83bc111110 con 0x7f83bc06fc60 2023-06-22T18:10:17.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.123+0000 7f83a37fe700 1 -- 172.21.15.39:0/189468544 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83bc111110 con 0x7f83bc06fc60 2023-06-22T18:10:17.127 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-06-22T18:10:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f83a409b680 msgr2=0x7f83a409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f83a409b680 0x7f83a409db30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f83bc1112b0 tx=0x7f83b000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 msgr2=0x7f83bc115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc115a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009b80 tx=0x7f83ac009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 shutdown_connections 2023-06-22T18:10:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f83a409b680 0x7f83a409db30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83bc1096b0 0x7f83bc10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83bc06fc60 0x7f83bc115a80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 --2- 172.21.15.39:0/189468544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83bc070570 0x7f83bc115fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 >> 172.21.15.39:0/189468544 conn(0x7f83bc074af0 msgr2=0x7f83bc0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.125+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 shutdown_connections 2023-06-22T18:10:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:17.126+0000 7f83c1606700 1 -- 172.21.15.39:0/189468544 wait complete. 2023-06-22T18:10:17.149 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:16 smithi039 ceph-mon[109976]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:17.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:16 smithi133 ceph-mon[116867]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:17.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:16 smithi101 ceph-mon[116267]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:17.662 INFO:tasks.cephadm:Adding mds.a on smithi039 2023-06-22T18:10:17.662 INFO:tasks.cephadm:Adding mds.d on smithi039 2023-06-22T18:10:17.663 INFO:tasks.cephadm:Adding mds.g on smithi039 2023-06-22T18:10:17.663 INFO:tasks.cephadm:Adding mds.j on smithi039 2023-06-22T18:10:17.664 INFO:tasks.cephadm:Adding mds.b on smithi101 2023-06-22T18:10:17.664 INFO:tasks.cephadm:Adding mds.e on smithi101 2023-06-22T18:10:17.665 INFO:tasks.cephadm:Adding mds.h on smithi101 2023-06-22T18:10:17.665 INFO:tasks.cephadm:Adding mds.k on smithi101 2023-06-22T18:10:17.666 INFO:tasks.cephadm:Adding mds.c on smithi133 2023-06-22T18:10:17.666 INFO:tasks.cephadm:Adding mds.f on smithi133 2023-06-22T18:10:17.666 INFO:tasks.cephadm:Adding mds.i on smithi133 2023-06-22T18:10:17.667 INFO:tasks.cephadm:Adding mds.l on smithi133 2023-06-22T18:10:17.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:42ddc46e6e6582d8e29a5a982a81886fe028be2c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d572fea0-1126-11ee-9b2b-001a4aab830c -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l' 2023-06-22T18:10:17.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:17 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/189468544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:18.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:17 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/189468544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:18.259 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-22T18:10:18.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:17 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/189468544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:19.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:18 smithi039 ceph-mon[109976]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:19.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:18 smithi133 ceph-mon[116867]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:19.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:18 smithi101 ceph-mon[116267]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/2286233384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f080fc5f0 msgr2=0x7f7f08104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/2286233384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f080fc5f0 0x7f7f08104090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009d60 tx=0x7f7f00004d10 comp rx=0 tx=0).stop 2023-06-22T18:10:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/2286233384 shutdown_connections 2023-06-22T18:10:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/2286233384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f080fc5f0 0x7f7f08104090 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/2286233384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7f080fd120 0x7f7f080fd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/2286233384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080fda70 0x7f7f080fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/2286233384 >> 172.21.15.133:0/2286233384 conn(0x7f7f080f6a50 msgr2=0x7f7f080f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/2286233384 shutdown_connections 2023-06-22T18:10:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/2286233384 wait complete. 2023-06-22T18:10:19.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 Processor -- start 2023-06-22T18:10:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 -- start start 2023-06-22T18:10:19.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.598+0000 7f7f0dbe0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080fc5f0 0x7f7f080fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7f080fd120 0x7f7f081124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 0x7f7f08114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f081156a0 con 0x7f7f080fc5f0 2023-06-22T18:10:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08115810 con 0x7f7f080fd120 2023-06-22T18:10:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f081159d0 con 0x7f7f08112a80 2023-06-22T18:10:19.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 0x7f7f08114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 0x7f7f08114f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47972/0 (socket says 172.21.15.133:47972) 2023-06-22T18:10:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 -- 172.21.15.133:0/3142939300 learned_addr learned my addr 172.21.15.133:0/3142939300 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-06-22T18:10:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f06ffd700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7f080fd120 0x7f7f081124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f077fe700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080fc5f0 0x7f7f080fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 -- 172.21.15.133:0/3142939300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7f080fd120 msgr2=0x7f7f081124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7f080fd120 0x7f7f081124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 -- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080fc5f0 msgr2=0x7f7f080fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080fc5f0 0x7f7f080fb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 -- 172.21.15.133:0/3142939300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00009940 con 0x7f7f08112a80 2023-06-22T18:10:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f07fff700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 0x7f7f08114f00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f000091e0 tx=0x7f7f00004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000e070 con 0x7f7f08112a80 2023-06-22T18:10:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f00012d00 con 0x7f7f08112a80 2023-06-22T18:10:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f08115c50 con 0x7f7f08112a80 2023-06-22T18:10:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0001cc90 con 0x7f7f08112a80 2023-06-22T18:10:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.599+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f08116000 con 0x7f7f08112a80 2023-06-22T18:10:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.600+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f08102870 con 0x7f7f08112a80 2023-06-22T18:10:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.601+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f7f0001c440 con 0x7f7f08112a80 2023-06-22T18:10:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.602+0000 7f7eeffff700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7ef009b680 0x7f7ef009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.602+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f7f000c7610 con 0x7f7f08112a80 2023-06-22T18:10:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.602+0000 7f7f077fe700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7ef009b680 0x7f7ef009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.603+0000 7f7f077fe700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7ef009b680 0x7f7ef009db30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8007fd0 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:19.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.605+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f7f00093270 con 0x7f7f08112a80 2023-06-22T18:10:19.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.795+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 --> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f0805df70 con 0x7f7ef009b680 2023-06-22T18:10:19.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.804+0000 7f7eeffff700 1 -- 172.21.15.133:0/3142939300 <== mgr.14152 v2:172.21.15.39:6800/2143792276 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f7f0805df70 con 0x7f7ef009b680 2023-06-22T18:10:19.807 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mds.all update... 2023-06-22T18:10:19.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7ef009b680 msgr2=0x7f7ef009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:19.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7ef009b680 0x7f7ef009db30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8007fd0 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2023-06-22T18:10:19.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 msgr2=0x7f7f08114f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:19.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 0x7f7f08114f00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7f000091e0 tx=0x7f7f00004d10 comp rx=0 tx=0).stop 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 shutdown_connections 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7ef009b680 0x7f7ef009db30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f08112a80 0x7f7f08114f00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080fc5f0 0x7f7f080fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 --2- 172.21.15.133:0/3142939300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7f080fd120 0x7f7f081124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 >> 172.21.15.133:0/3142939300 conn(0x7f7f080f6a50 msgr2=0x7f7f080f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:19.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 shutdown_connections 2023-06-22T18:10:19.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-06-22T18:10:19.806+0000 7f7f0dbe0700 1 -- 172.21.15.133:0/3142939300 wait complete. 2023-06-22T18:10:20.492 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.a.service 2023-06-22T18:10:20.498 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.d.service 2023-06-22T18:10:20.544 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.g.service 2023-06-22T18:10:20.567 INFO:journalctl@ceph.mds.a.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:10:20.570 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.j.service 2023-06-22T18:10:20.607 INFO:journalctl@ceph.mds.d.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:10:20.610 DEBUG:teuthology.orchestra.run.smithi101:mds.b> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.b.service 2023-06-22T18:10:20.613 DEBUG:teuthology.orchestra.run.smithi101:mds.e> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.e.service 2023-06-22T18:10:20.632 INFO:journalctl@ceph.mds.g.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:10:20.641 INFO:journalctl@ceph.mds.b.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:10:20.641 DEBUG:teuthology.orchestra.run.smithi101:mds.h> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.h.service 2023-06-22T18:10:20.664 INFO:journalctl@ceph.mds.e.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:10:20.665 DEBUG:teuthology.orchestra.run.smithi101:mds.k> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.k.service 2023-06-22T18:10:20.672 INFO:journalctl@ceph.mds.j.smithi039.stdout:-- Logs begin at Thu 2023-06-22 17:40:36 UTC. -- 2023-06-22T18:10:20.687 INFO:journalctl@ceph.mds.h.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:10:20.688 DEBUG:teuthology.orchestra.run.smithi133:mds.c> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.c.service 2023-06-22T18:10:20.690 DEBUG:teuthology.orchestra.run.smithi133:mds.f> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.f.service 2023-06-22T18:10:20.710 INFO:journalctl@ceph.mds.k.smithi101.stdout:-- Logs begin at Thu 2023-06-22 17:35:06 UTC. -- 2023-06-22T18:10:20.717 INFO:journalctl@ceph.mds.c.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:10:20.718 DEBUG:teuthology.orchestra.run.smithi133:mds.i> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.i.service 2023-06-22T18:10:20.742 INFO:journalctl@ceph.mds.f.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:10:20.743 DEBUG:teuthology.orchestra.run.smithi133:mds.l> sudo journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.l.service 2023-06-22T18:10:20.767 INFO:journalctl@ceph.mds.i.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:10:20.768 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-06-22T18:10:20.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T18:10:20.792 INFO:journalctl@ceph.mds.l.smithi133.stdout:-- Logs begin at Thu 2023-06-22 17:35:05 UTC. -- 2023-06-22T18:10:20.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:20.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:10:20.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2023-06-22T18:10:20.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:20.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:20.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:20.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:20.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:20.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:20.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:20.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:20 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 -- 172.21.15.39:0/1560974062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe638071950 msgr2=0x7fe63806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 --2- 172.21.15.39:0/1560974062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe638071950 0x7fe63806fdc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe628004900 tx=0x7fe628009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 -- 172.21.15.39:0/1560974062 shutdown_connections 2023-06-22T18:10:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 --2- 172.21.15.39:0/1560974062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6381049f0 0x7fe638106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 --2- 172.21.15.39:0/1560974062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe638071950 0x7fe63806fdc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 --2- 172.21.15.39:0/1560974062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 0x7fe638070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 -- 172.21.15.39:0/1560974062 >> 172.21.15.39:0/1560974062 conn(0x7fe6380f8230 msgr2=0x7fe6380fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.982+0000 7fe637fff700 1 -- 172.21.15.39:0/1560974062 shutdown_connections 2023-06-22T18:10:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.983+0000 7fe637fff700 1 -- 172.21.15.39:0/1560974062 wait complete. 2023-06-22T18:10:20.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.983+0000 7fe637fff700 1 Processor -- start 2023-06-22T18:10:20.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.983+0000 7fe637fff700 1 -- start start 2023-06-22T18:10:20.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe637fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 0x7fe63810f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:20.990 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe637fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 0x7fe63810fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe637fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6381049f0 0x7fe638116f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe637fff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe638117580 con 0x7fe6381049f0 2023-06-22T18:10:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe637fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6381176f0 con 0x7fe638071950 2023-06-22T18:10:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe637fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6381178b0 con 0x7fe638070300 2023-06-22T18:10:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 0x7fe63810fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:20.991 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 0x7fe63810fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:41870/0 (socket says 172.21.15.39:41870) 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 -- 172.21.15.39:0/4197088214 learned_addr learned my addr 172.21.15.39:0/4197088214 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe636ffd700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 0x7fe63810f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6377fe700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6381049f0 0x7fe638116f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 -- 172.21.15.39:0/4197088214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 msgr2=0x7fe63810f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 0x7fe63810f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 -- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6381049f0 msgr2=0x7fe638116f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6381049f0 0x7fe638116f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe6367fc700 1 -- 172.21.15.39:0/4197088214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe628005040 con 0x7fe638071950 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.984+0000 7fe636ffd700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 0x7fe63810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.985+0000 7fe6367fc700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 0x7fe63810fee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0041d0 tx=0x7fe62c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.985+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c007cc0 con 0x7fe638071950 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.985+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe62c00f040 con 0x7fe638071950 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.985+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe638117b30 con 0x7fe638071950 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.985+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c00ca00 con 0x7fe638071950 2023-06-22T18:10:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.985+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe638118070 con 0x7fe638071950 2023-06-22T18:10:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.986+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe638103140 con 0x7fe638071950 2023-06-22T18:10:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.988+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fe62c0090d0 con 0x7fe638071950 2023-06-22T18:10:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.989+0000 7fe61f7fe700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe62009b700 0x7fe62009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.989+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fe62c0c4360 con 0x7fe638071950 2023-06-22T18:10:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.989+0000 7fe636ffd700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe62009b700 0x7fe62009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.990+0000 7fe636ffd700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe62009b700 0x7fe62009dbb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe628004f70 tx=0x7fe628009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:20.995 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:20.993+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fe62c04da00 con 0x7fe638071950 2023-06-22T18:10:21.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:21.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:10:21.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2023-06-22T18:10:21.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:21.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:21.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:21.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:21.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:21.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:21.105 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:21.106 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:20 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:21.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:21.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-22T18:10:21.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:21.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:20 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:21.201 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.199+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe63802b1d0 con 0x7fe638071950 2023-06-22T18:10:21.203 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.200+0000 7fe61f7fe700 1 -- 172.21.15.39:0/4197088214 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18120 (secure 0 0 0) 0x7fe62c08ffc0 con 0x7fe638071950 2023-06-22T18:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:21.204 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","created":"2023-06-22T18:02:53.160326+0000","modified":"2023-06-22T18:10:12.224478+0000","last_up_change":"2023-06-22T18:10:10.873966+0000","last_in_change":"2023-06-22T18:09:51.312474+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T18:06:20.190713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f55d12a1-86b5-42a6-ad35-d5eaf3d99c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6803","nonce":471291770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6805","nonce":471291770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6809","nonce":471291770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6807","nonce":471291770}]},"public_addr":"172.21.15.39:6803/471291770","cluster_addr":"172.21.15.39:6805/471291770","heartbeat_back_addr":"172.21.15.39:6809/471291770","heartbeat_front_addr":"172.21.15.39:6807/471291770","state":["exists","up"]},{"osd":1,"uuid":"4d8fb427-c4b2-45c3-ac42-dea08c42e4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6801","nonce":840111728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6803","nonce":840111728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6807","nonce":840111728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6805","nonce":840111728}]},"public_addr":"172.21.15.101:6801/840111728","cluster_addr":"172.21.15.101:6803/840111728","heartbeat_back_addr":"172.21.15.101:6807/840111728","heartbeat_front_addr":"172.21.15.101:6805/840111728","state":["exists","up"]},{"osd":2,"uuid":"9c714cec-14fc-4aed-b3f1-f0773edd197c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6801","nonce":3165338416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6803","nonce":3165338416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6807","nonce":3165338416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6805","nonce":3165338416}]},"public_addr":"172.21.15.133:6801/3165338416","cluster_addr":"172.21.15.133:6803/3165338416","heartbeat_back_addr":"172.21.15.133:6807/3165338416","heartbeat_front_addr":"172.21.15.133:6805/3165338416","state":["exists","up"]},{"osd":3,"uuid":"71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6811","nonce":3695104438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6813","nonce":3695104438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6817","nonce":3695104438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6815","nonce":3695104438}]},"public_addr":"172.21.15.39:6811/3695104438","cluster_addr":"172.21.15.39:6813/3695104438","heartbeat_back_addr":"172.21.15.39:6817/3695104438","heartbeat_front_addr":"172.21.15.39:6815/3695104438","state":["exists","up"]},{"osd":4,"uuid":"c9ecace5-1032-4f25-a5f3-84ec819c83ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6809","nonce":2726745315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6811","nonce":2726745315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6815","nonce":2726745315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6813","nonce":2726745315}]},"public_addr":"172.21.15.101:6809/2726745315","cluster_addr":"172.21.15.101:6811/2726745315","heartbeat_back_addr":"172.21.15.101:6815/2726745315","heartbeat_front_addr":"172.21.15.101:6813/2726745315","state":["exists","up"]},{"osd":5,"uuid":"379ed499-68d9-4032-bafd-31cb02995388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6809","nonce":1467210441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6811","nonce":1467210441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6815","nonce":1467210441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6813","nonce":1467210441}]},"public_addr":"172.21.15.133:6809/1467210441","cluster_addr":"172.21.15.133:6811/1467210441","heartbeat_back_addr":"172.21.15.133:6815/1467210441","heartbeat_front_addr":"172.21.15.133:6813/1467210441","state":["exists","up"]},{"osd":6,"uuid":"9cf1482c-6b66-4260-a892-da6969cc8574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6819","nonce":4262819120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6821","nonce":4262819120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6825","nonce":4262819120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6823","nonce":4262819120}]},"public_addr":"172.21.15.39:6819/4262819120","cluster_addr":"172.21.15.39:6821/4262819120","heartbeat_back_addr":"172.21.15.39:6825/4262819120","heartbeat_front_addr":"172.21.15.39:6823/4262819120","state":["exists","up"]},{"osd":7,"uuid":"7b535b3c-1d36-44c9-8c22-132e6d31b0ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6817","nonce":3667650401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6819","nonce":3667650401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6823","nonce":3667650401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6821","nonce":3667650401}]},"public_addr":"172.21.15.101:6817/3667650401","cluster_addr":"172.21.15.101:6819/3667650401","heartbeat_back_addr":"172.21.15.101:6823/3667650401","heartbeat_front_addr":"172.21.15.101:6821/3667650401","state":["exists","up"]},{"osd":8,"uuid":"60376342-edba-4eb8-9907-950963e6c3e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6817","nonce":1333554117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6819","nonce":1333554117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6823","nonce":1333554117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6821","nonce":1333554117}]},"public_addr":"172.21.15.133:6817/1333554117","cluster_addr":"172.21.15.133:6819/1333554117","heartbeat_back_addr":"172.21.15.133:6823/1333554117","heartbeat_front_addr":"172.21.15.133:6821/1333554117","state":["exists","up"]},{"osd":9,"uuid":"c09c5c50-6e36-4e8e-b41c-50fb93a2ea92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6827","nonce":881304983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6829","nonce":881304983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6833","nonce":881304983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6831","nonce":881304983}]},"public_addr":"172.21.15.39:6827/881304983","cluster_addr":"172.21.15.39:6829/881304983","heartbeat_back_addr":"172.21.15.39:6833/881304983","heartbeat_front_addr":"172.21.15.39:6831/881304983","state":["exists","up"]},{"osd":10,"uuid":"4048ac5c-ffb1-4b04-85ec-6afc2add5b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6825","nonce":1823054778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6827","nonce":1823054778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6831","nonce":1823054778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6829","nonce":1823054778}]},"public_addr":"172.21.15.101:6825/1823054778","cluster_addr":"172.21.15.101:6827/1823054778","heartbeat_back_addr":"172.21.15.101:6831/1823054778","heartbeat_front_addr":"172.21.15.101:6829/1823054778","state":["exists","up"]},{"osd":11,"uuid":"334df2ca-0d0e-4450-bd11-cd8651c44e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6825","nonce":93348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6827","nonce":93348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6831","nonce":93348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6829","nonce":93348943}]},"public_addr":"172.21.15.133:6825/93348943","cluster_addr":"172.21.15.133:6827/93348943","heartbeat_back_addr":"172.21.15.133:6831/93348943","heartbeat_front_addr":"172.21.15.133:6829/93348943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:27.345253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:52.437673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:17.253223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:44.552567+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:09.454586+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:34.594288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:01.497366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:26.643829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:51.455308+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:18.528459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:43.697768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:10:09.419715+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/709665139":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/453675837":"2023-06-23T18:03:53.598476+0000","172.21.15.39:6801/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6800/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/2777224344":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/3733013426":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/466669289":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6801/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/1167071234":"2023-06-23T18:03:53.598476+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe62009b700 msgr2=0x7fe62009dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe62009b700 0x7fe62009dbb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe628004f70 tx=0x7fe628009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 msgr2=0x7fe63810fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 0x7fe63810fee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0041d0 tx=0x7fe62c00b040 comp rx=0 tx=0).stop 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 shutdown_connections 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe62009b700 0x7fe62009dbb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.206 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6381049f0 0x7fe638116f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe638070300 0x7fe63810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 --2- 172.21.15.39:0/4197088214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe638071950 0x7fe63810fee0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 >> 172.21.15.39:0/4197088214 conn(0x7fe6380f8230 msgr2=0x7fe6380fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:21.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 shutdown_connections 2023-06-22T18:10:21.207 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.202+0000 7fe637fff700 1 -- 172.21.15.39:0/4197088214 wait complete. 2023-06-22T18:10:21.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-22T18:06:20.190713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-22T18:10:21.225 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-22T18:10:21.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 -- 172.21.15.39:0/864214625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 msgr2=0x7fcca80fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.416 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 --2- 172.21.15.39:0/864214625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca80fe140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc980049c0 tx=0x7fcc98009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 -- 172.21.15.39:0/864214625 shutdown_connections 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 --2- 172.21.15.39:0/864214625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 --2- 172.21.15.39:0/864214625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 0x7fcca80fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 --2- 172.21.15.39:0/864214625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca80fe140 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 -- 172.21.15.39:0/864214625 >> 172.21.15.39:0/864214625 conn(0x7fcca80f8240 msgr2=0x7fcca80fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 -- 172.21.15.39:0/864214625 shutdown_connections 2023-06-22T18:10:21.417 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.413+0000 7fccae99d700 1 -- 172.21.15.39:0/864214625 wait complete. 2023-06-22T18:10:21.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.414+0000 7fccae99d700 1 Processor -- start 2023-06-22T18:10:21.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.414+0000 7fccae99d700 1 -- start start 2023-06-22T18:10:21.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.414+0000 7fccae99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 0x7fcca810d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.419 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae99d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca810dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae99d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8114d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae99d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8115310 con 0x7fcca80fe680 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8115480 con 0x7fcca80fcae0 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae99d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8115640 con 0x7fcca80fdcf0 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8114d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccad19a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca810dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.420 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8114d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47846/0 (socket says 172.21.15.39:47846) 2023-06-22T18:10:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccad19a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca810dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:38882/0 (socket says 172.21.15.39:38882) 2023-06-22T18:10:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 -- 172.21.15.39:0/1888814013 learned_addr learned my addr 172.21.15.39:0/1888814013 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccad99b700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 0x7fcca810d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 -- 172.21.15.39:0/1888814013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 msgr2=0x7fcca810dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca810dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.421 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 -- 172.21.15.39:0/1888814013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 msgr2=0x7fcca810d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 0x7fcca810d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 -- 172.21.15.39:0/1888814013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca4005040 con 0x7fcca80fe680 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccad19a700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccad99b700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 0x7fcca810d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.415+0000 7fccae19c700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8114d00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcca000ec30 tx=0x7fcca000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.416+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0007b30 con 0x7fcca80fe680 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.416+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc98005040 con 0x7fcca80fe680 2023-06-22T18:10:21.422 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.416+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcca000f040 con 0x7fcca80fe680 2023-06-22T18:10:21.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.416+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca000ca30 con 0x7fcca80fe680 2023-06-22T18:10:21.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.416+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca8115d00 con 0x7fcca80fe680 2023-06-22T18:10:21.423 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.417+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8103950 con 0x7fcca80fe680 2023-06-22T18:10:21.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.419+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fcca001d400 con 0x7fcca80fe680 2023-06-22T18:10:21.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.421+0000 7fcc9e7fc700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcc9409b700 0x7fcc9409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.421+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fcca00c4e10 con 0x7fcca80fe680 2023-06-22T18:10:21.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.421+0000 7fccad99b700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcc9409b700 0x7fcc9409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.424 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.421+0000 7fccad99b700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcc9409b700 0x7fcc9409dbb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcca80fdb50 tx=0x7fcca400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:21.427 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.425+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fcca00c8030 con 0x7fcca80fe680 2023-06-22T18:10:21.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.628+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcca802b180 con 0x7fcca80fe680 2023-06-22T18:10:21.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.629+0000 7fcc9e7fc700 1 -- 172.21.15.39:0/1888814013 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcca0090a70 con 0x7fcca80fe680 2023-06-22T18:10:21.632 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-06-22T18:10:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcc9409b700 msgr2=0x7fcc9409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcc9409b700 0x7fcc9409dbb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcca80fdb50 tx=0x7fcca400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 msgr2=0x7fcca8114d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8114d00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcca000ec30 tx=0x7fcca000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 shutdown_connections 2023-06-22T18:10:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcc9409b700 0x7fcc9409dbb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80fe680 0x7fcca8114d00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcca80fcae0 0x7fcca810d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 --2- 172.21.15.39:0/1888814013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcca80fdcf0 0x7fcca810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.631+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 >> 172.21.15.39:0/1888814013 conn(0x7fcca80f8240 msgr2=0x7fcca80fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.632+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 shutdown_connections 2023-06-22T18:10:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.632+0000 7fccae99d700 1 -- 172.21.15.39:0/1888814013 wait complete. 2023-06-22T18:10:21.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:21.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.838+0000 7f610c7ff700 1 -- 172.21.15.39:0/554501013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fe930 msgr2=0x7f61041051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.838+0000 7f610c7ff700 1 --2- 172.21.15.39:0/554501013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fe930 0x7f61041051d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d60 tx=0x7f60fc004d10 comp rx=0 tx=0).stop 2023-06-22T18:10:21.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 -- 172.21.15.39:0/554501013 shutdown_connections 2023-06-22T18:10:21.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 --2- 172.21.15.39:0/554501013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fe930 0x7f61041051d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.845 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 --2- 172.21.15.39:0/554501013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fccd0 0x7f61040fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 --2- 172.21.15.39:0/554501013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61040fdf80 0x7f61040fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 -- 172.21.15.39:0/554501013 >> 172.21.15.39:0/554501013 conn(0x7f61040f83f0 msgr2=0x7f61040fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 -- 172.21.15.39:0/554501013 shutdown_connections 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 -- 172.21.15.39:0/554501013 wait complete. 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.839+0000 7f610c7ff700 1 Processor -- start 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 -- start start 2023-06-22T18:10:21.846 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61040fccd0 0x7f610410d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 0x7f610410dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fe930 0x7f6104114d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104115330 con 0x7f61040fdf80 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61041154a0 con 0x7f61040fccd0 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610c7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6104115660 con 0x7f61040fe930 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f6109d9a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 0x7f610410dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610ad9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fe930 0x7f6104114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f6109d9a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 0x7f610410dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47870/0 (socket says 172.21.15.39:47870) 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f6109d9a700 1 -- 172.21.15.39:0/3929319525 learned_addr learned my addr 172.21.15.39:0/3929319525 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610a59b700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61040fccd0 0x7f610410d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610ad9c700 1 -- 172.21.15.39:0/3929319525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61040fccd0 msgr2=0x7f610410d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.840+0000 7f610ad9c700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61040fccd0 0x7f610410d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f610ad9c700 1 -- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 msgr2=0x7f610410dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f610ad9c700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 0x7f610410dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:21.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f610ad9c700 1 -- 172.21.15.39:0/3929319525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc009940 con 0x7f61040fe930 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f6109d9a700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 0x7f610410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f610ad9c700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fe930 0x7f6104114d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d60 tx=0x7f60fc004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc00e030 con 0x7f61040fe930 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61041158e0 con 0x7f61040fe930 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60fc00bec0 con 0x7f61040fe930 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc012410 con 0x7f61040fe930 2023-06-22T18:10:21.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.841+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6104115e10 con 0x7f61040fe930 2023-06-22T18:10:21.850 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.842+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6104108530 con 0x7f61040fe930 2023-06-22T18:10:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.844+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f60fc01c650 con 0x7f61040fe930 2023-06-22T18:10:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.845+0000 7f60faffd700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f60f009b700 0x7f60f009dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.845+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f60fc0c6f90 con 0x7f61040fe930 2023-06-22T18:10:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.845+0000 7f610a59b700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f60f009b700 0x7f60f009dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.846+0000 7f610a59b700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f60f009b700 0x7f60f009dbb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6100009b80 tx=0x7f610000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:21.851 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:21.847+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f60fc092bf0 con 0x7f61040fe930 2023-06-22T18:10:21.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:21 smithi039 ceph-mon[109976]: Deploying daemon mds.l on smithi133 2023-06-22T18:10:21.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:21 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/4197088214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:21.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:21 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1888814013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:22.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:21 smithi133 ceph-mon[116867]: Deploying daemon mds.l on smithi133 2023-06-22T18:10:22.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:21 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/4197088214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:22.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:21 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1888814013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.098+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61041086c0 con 0x7f61040fe930 2023-06-22T18:10:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.098+0000 7f60faffd700 1 -- 172.21.15.39:0/3929319525 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f61041086c0 con 0x7f61040fe930 2023-06-22T18:10:22.102 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:22.102 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-06-22T18:10:22.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:21 smithi101 ceph-mon[116267]: Deploying daemon mds.l on smithi133 2023-06-22T18:10:22.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:21 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/4197088214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:22.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:21 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1888814013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f60f009b700 msgr2=0x7f60f009dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f60f009b700 0x7f60f009dbb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6100009b80 tx=0x7f610000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fe930 msgr2=0x7f6104114d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fe930 0x7f6104114d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d60 tx=0x7f60fc004d10 comp rx=0 tx=0).stop 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 shutdown_connections 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f60f009b700 0x7f60f009dbb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61040fe930 0x7f6104114d20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f61040fccd0 0x7f610410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 --2- 172.21.15.39:0/3929319525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61040fdf80 0x7f610410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 >> 172.21.15.39:0/3929319525 conn(0x7f61040f83f0 msgr2=0x7f61040fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.100+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 shutdown_connections 2023-06-22T18:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:22.101+0000 7f610c7ff700 1 -- 172.21.15.39:0/3929319525 wait complete. 2023-06-22T18:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 1 2023-06-22T18:10:22.340 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:10:22 smithi133 systemd[1]: Starting Ceph mds.l for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:22.734 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mds[149493]: -- 172.21.15.133:0/1146127127 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5572a177e9c0 con 0x5572a1755000 2023-06-22T18:10:22.734 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:10:22 smithi133 conmon[149468]: starting mds.l at 2023-06-22T18:10:22.734 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:10:22 smithi133 bash[149428]: 4e07cc823edd359d57744aafc2abc8531de4b23ba2a30d51d893099d8d6a452f 2023-06-22T18:10:22.734 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:10:22 smithi133 systemd[1]: Started Ceph mds.l for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:23.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3929319525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.133:6832/2685285663,v1:172.21.15.133:6833/2685285663] up:boot 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: fsmap 1 up:standby 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:23.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:23.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:22 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3929319525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.133:6832/2685285663,v1:172.21.15.133:6833/2685285663] up:boot 2023-06-22T18:10:23.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: fsmap 1 up:standby 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:23.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:22 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3929319525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.133:6832/2685285663,v1:172.21.15.133:6833/2685285663] up:boot 2023-06-22T18:10:23.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: fsmap 1 up:standby 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:23.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:22 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:24.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:23 smithi101 ceph-mon[116267]: Deploying daemon mds.k on smithi101 2023-06-22T18:10:24.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:24.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:23 smithi039 ceph-mon[109976]: Deploying daemon mds.k on smithi101 2023-06-22T18:10:24.233 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:23 smithi133 ceph-mon[116867]: Deploying daemon mds.k on smithi101 2023-06-22T18:10:24.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2151152316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fe6a0 msgr2=0x7f18dc104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:24.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2151152316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fe6a0 0x7f18dc104f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d60 tx=0x7f18d4004d10 comp rx=0 tx=0).stop 2023-06-22T18:10:24.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2151152316 shutdown_connections 2023-06-22T18:10:24.323 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2151152316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fe6a0 0x7f18dc104f60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2151152316 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18dc0fcb00 0x7f18dc0fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2151152316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18dc0fdd10 0x7f18dc0fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2151152316 >> 172.21.15.39:0/2151152316 conn(0x7f18dc0f8240 msgr2=0x7f18dc0fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2151152316 shutdown_connections 2023-06-22T18:10:24.324 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.319+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2151152316 wait complete. 2023-06-22T18:10:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.320+0000 7f18e0f9d700 1 Processor -- start 2023-06-22T18:10:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.320+0000 7f18e0f9d700 1 -- start start 2023-06-22T18:10:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.320+0000 7f18e0f9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 0x7f18dc10f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.320+0000 7f18e0f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18dc0fdd10 0x7f18dc10fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18e0f9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18dc0fe6a0 0x7f18dc116da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:24.326 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18e0f9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1173b0 con 0x7f18dc0fcb00 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18e0f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc117520 con 0x7f18dc0fdd10 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18e0f9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1176e0 con 0x7f18dc0fe6a0 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 0x7f18dc10f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 0x7f18dc10f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47890/0 (socket says 172.21.15.39:47890) 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 -- 172.21.15.39:0/2814408933 learned_addr learned my addr 172.21.15.39:0/2814408933 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18daffd700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18dc0fdd10 0x7f18dc10fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18dbfff700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18dc0fe6a0 0x7f18dc116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 -- 172.21.15.39:0/2814408933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18dc0fe6a0 msgr2=0x7f18dc116da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18dc0fe6a0 0x7f18dc116da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 -- 172.21.15.39:0/2814408933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18dc0fdd10 msgr2=0x7f18dc10fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18dc0fdd10 0x7f18dc10fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 -- 172.21.15.39:0/2814408933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009940 con 0x7f18dc0fcb00 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.321+0000 7f18db7fe700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 0x7f18dc10f7d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00b810 tx=0x7f18cc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.322+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00f510 con 0x7f18dc0fcb00 2023-06-22T18:10:24.328 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.322+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18cc00fb40 con 0x7f18dc0fcb00 2023-06-22T18:10:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.322+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc1179c0 con 0x7f18dc0fcb00 2023-06-22T18:10:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.322+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc011c30 con 0x7f18dc0fcb00 2023-06-22T18:10:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.322+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc117f00 con 0x7f18dc0fcb00 2023-06-22T18:10:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.323+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc108390 con 0x7f18dc0fcb00 2023-06-22T18:10:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.325+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f18cc00fcb0 con 0x7f18dc0fcb00 2023-06-22T18:10:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.326+0000 7f18c3fff700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f18c409b700 0x7f18c409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.326+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f18cc0c4580 con 0x7f18dc0fcb00 2023-06-22T18:10:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.326+0000 7f18daffd700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f18c409b700 0x7f18c409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:24.330 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.327+0000 7f18daffd700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f18c409b700 0x7f18c409dbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009b80 tx=0x7f18d000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.330+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f18cc0901e0 con 0x7f18dc0fcb00 2023-06-22T18:10:24.550 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:10:24 smithi101 systemd[1]: Starting Ceph mds.k for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.581+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18dc108520 con 0x7f18dc0fcb00 2023-06-22T18:10:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.582+0000 7f18c3fff700 1 -- 172.21.15.39:0/2814408933 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f18dc108520 con 0x7f18dc0fcb00 2023-06-22T18:10:24.585 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:24.586 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-22T18:10:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f18c409b700 msgr2=0x7f18c409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f18c409b700 0x7f18c409dbb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009b80 tx=0x7f18d000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 msgr2=0x7f18dc10f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 0x7f18dc10f7d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00b810 tx=0x7f18cc009220 comp rx=0 tx=0).stop 2023-06-22T18:10:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 shutdown_connections 2023-06-22T18:10:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f18c409b700 0x7f18c409dbb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18dc0fe6a0 0x7f18dc116da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0fcb00 0x7f18dc10f7d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 --2- 172.21.15.39:0/2814408933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18dc0fdd10 0x7f18dc10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 >> 172.21.15.39:0/2814408933 conn(0x7f18dc0f8240 msgr2=0x7f18dc06fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 shutdown_connections 2023-06-22T18:10:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:24.584+0000 7f18e0f9d700 1 -- 172.21.15.39:0/2814408933 wait complete. 2023-06-22T18:10:24.590 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2023-06-22T18:10:24.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:24 smithi039 ceph-mon[109976]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:24.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:24 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2814408933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:24 smithi101 ceph-mon[116267]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:25.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:24 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2814408933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:25.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:24 smithi133 ceph-mon[116867]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:25.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:24 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2814408933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:25.602 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:10:25 smithi101 ceph-mds[147242]: -- 172.21.15.101:0/3567982594 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d494dc89c0 con 0x55d494d9f000 2023-06-22T18:10:25.603 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:10:25 smithi101 conmon[147217]: starting mds.k at 2023-06-22T18:10:25.603 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:10:25 smithi101 bash[147177]: a0798a741cd8174dc7c71c35e41a830899fed94c3c39c249a288fd60f70592b4 2023-06-22T18:10:25.603 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:10:25 smithi101 systemd[1]: Started Ceph mds.k for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:26.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: Deploying daemon mds.j on smithi039 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.101:6832/4229656332,v1:172.21.15.101:6833/4229656332] up:boot 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: fsmap 2 up:standby 2023-06-22T18:10:26.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:26 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-22T18:10:26.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:26.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:26.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:26.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: Deploying daemon mds.j on smithi039 2023-06-22T18:10:26.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:26.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.101:6832/4229656332,v1:172.21.15.101:6833/4229656332] up:boot 2023-06-22T18:10:26.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: fsmap 2 up:standby 2023-06-22T18:10:26.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:26 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-22T18:10:26.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:26.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:26.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:26.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:26.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: Deploying daemon mds.j on smithi039 2023-06-22T18:10:26.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:26.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.101:6832/4229656332,v1:172.21.15.101:6833/4229656332] up:boot 2023-06-22T18:10:26.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: fsmap 2 up:standby 2023-06-22T18:10:26.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:26 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-22T18:10:26.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:26.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.796+0000 7f7b575ff700 1 -- 172.21.15.39:0/3145743612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fdf80 msgr2=0x7f7b500fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:26.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.796+0000 7f7b575ff700 1 --2- 172.21.15.39:0/3145743612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fdf80 0x7f7b500fe3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b400049c0 tx=0x7f7b40009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:26.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 -- 172.21.15.39:0/3145743612 shutdown_connections 2023-06-22T18:10:26.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 --2- 172.21.15.39:0/3145743612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7b500fe930 0x7f7b501051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:26.801 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 --2- 172.21.15.39:0/3145743612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b500fccd0 0x7f7b500fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:26.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 --2- 172.21.15.39:0/3145743612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fdf80 0x7f7b500fe3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:26.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 -- 172.21.15.39:0/3145743612 >> 172.21.15.39:0/3145743612 conn(0x7f7b500f83f0 msgr2=0x7f7b500fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:26.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 -- 172.21.15.39:0/3145743612 shutdown_connections 2023-06-22T18:10:26.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 -- 172.21.15.39:0/3145743612 wait complete. 2023-06-22T18:10:26.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.797+0000 7f7b575ff700 1 Processor -- start 2023-06-22T18:10:26.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 -- start start 2023-06-22T18:10:26.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 0x7f7b50071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:26.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7b500fdf80 0x7f7b5006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:26.804 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b500fe930 0x7f7b500702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50070a30 con 0x7f7b500fccd0 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50070ba0 con 0x7f7b500fdf80 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b575ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50117aa0 con 0x7f7b500fe930 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b5539b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 0x7f7b50071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b5539b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 0x7f7b50071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47066/0 (socket says 172.21.15.39:47066) 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b5539b700 1 -- 172.21.15.39:0/855863369 learned_addr learned my addr 172.21.15.39:0/855863369 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b54b9a700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7b500fdf80 0x7f7b5006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:26.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b55b9c700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b500fe930 0x7f7b500702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:26.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.798+0000 7f7b5539b700 1 -- 172.21.15.39:0/855863369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b500fe930 msgr2=0x7f7b500702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:26.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b5539b700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b500fe930 0x7f7b500702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:26.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b5539b700 1 -- 172.21.15.39:0/855863369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7b500fdf80 msgr2=0x7f7b5006fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:26.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b5539b700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7b500fdf80 0x7f7b5006fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:26.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b5539b700 1 -- 172.21.15.39:0/855863369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40005040 con 0x7f7b500fccd0 2023-06-22T18:10:26.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b5539b700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 0x7f7b50071720 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00ebc0 tx=0x7f7b4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:26.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c007cc0 con 0x7f7b500fccd0 2023-06-22T18:10:26.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b4c00f040 con 0x7f7b500fccd0 2023-06-22T18:10:26.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0094a0 con 0x7f7b500fccd0 2023-06-22T18:10:26.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b50117cf0 con 0x7f7b500fccd0 2023-06-22T18:10:26.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.799+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b50118190 con 0x7f7b500fccd0 2023-06-22T18:10:26.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.800+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b50103c50 con 0x7f7b500fccd0 2023-06-22T18:10:26.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.802+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f7b4c00c600 con 0x7f7b500fccd0 2023-06-22T18:10:26.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.804+0000 7f7b45ffb700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7b3c09b700 0x7f7b3c09dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:26.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.804+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f7b4c0c4560 con 0x7f7b500fccd0 2023-06-22T18:10:26.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.804+0000 7f7b54b9a700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7b3c09b700 0x7f7b3c09dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:26.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.804+0000 7f7b54b9a700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7b3c09b700 0x7f7b3c09dbb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b40004890 tx=0x7f7b40009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:26.811 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:26.808+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f7b4c04da00 con 0x7f7b500fccd0 2023-06-22T18:10:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.101+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b5002b200 con 0x7f7b500fccd0 2023-06-22T18:10:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.101+0000 7f7b45ffb700 1 -- 172.21.15.39:0/855863369 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f7b4c019800 con 0x7f7b500fccd0 2023-06-22T18:10:27.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:27.106 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-22T18:10:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7b3c09b700 msgr2=0x7f7b3c09dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7b3c09b700 0x7f7b3c09dbb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b40004890 tx=0x7f7b40009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 msgr2=0x7f7b50071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 0x7f7b50071720 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00ebc0 tx=0x7f7b4c00b040 comp rx=0 tx=0).stop 2023-06-22T18:10:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 shutdown_connections 2023-06-22T18:10:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f7b3c09b700 0x7f7b3c09dbb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b500fe930 0x7f7b500702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b500fccd0 0x7f7b50071720 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 --2- 172.21.15.39:0/855863369 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7b500fdf80 0x7f7b5006fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 >> 172.21.15.39:0/855863369 conn(0x7f7b500f83f0 msgr2=0x7f7b500fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 shutdown_connections 2023-06-22T18:10:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:27.104+0000 7f7b575ff700 1 -- 172.21.15.39:0/855863369 wait complete. 2023-06-22T18:10:27.110 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2023-06-22T18:10:27.436 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:27 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/855863369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:27.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:27 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/855863369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:27.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:27 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/855863369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:27.944 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:10:27 smithi039 systemd[1]: Starting Ceph mds.j for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:28.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:28 smithi101 ceph-mon[116267]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:28.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:28 smithi039 ceph-mon[109976]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:28.694 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:10:28 smithi039 bash[152340]: 75f0a61a644d562a4deb9df2271dbccc08bb4822544f934d84e56020852660f0 2023-06-22T18:10:28.695 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:10:28 smithi039 ceph-mds[152405]: -- 172.21.15.39:0/3420283169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eaa13f29c0 con 0x55eaa13c8c00 2023-06-22T18:10:28.695 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:10:28 smithi039 conmon[152380]: starting mds.j at 2023-06-22T18:10:28.695 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:10:28 smithi039 systemd[1]: Started Ceph mds.j for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:28.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:28 smithi133 ceph-mon[116867]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:29.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:29.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 -- 172.21.15.39:0/3894786265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 msgr2=0x7ff4640fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:29.331 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3894786265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 0x7ff4640fe0e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4540049c0 tx=0x7ff454009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 -- 172.21.15.39:0/3894786265 shutdown_connections 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3894786265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fe620 0x7ff464104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3894786265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4640fca80 0x7ff4640fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3894786265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 0x7ff4640fe0e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 -- 172.21.15.39:0/3894786265 >> 172.21.15.39:0/3894786265 conn(0x7ff4640f81e0 msgr2=0x7ff4640fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 -- 172.21.15.39:0/3894786265 shutdown_connections 2023-06-22T18:10:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.327+0000 7ff46a39d700 1 -- 172.21.15.39:0/3894786265 wait complete. 2023-06-22T18:10:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.328+0000 7ff46a39d700 1 Processor -- start 2023-06-22T18:10:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.328+0000 7ff46a39d700 1 -- start start 2023-06-22T18:10:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46a39d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 0x7ff46410d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46a39d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 0x7ff46410dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46a39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4640fe620 0x7ff464114ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46a39d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4641152f0 con 0x7ff4640fca80 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46a39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff464115460 con 0x7ff4640fe620 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46a39d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff464115620 con 0x7ff4640fdc90 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 0x7ff46410d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 0x7ff46410d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47100/0 (socket says 172.21.15.39:47100) 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 -- 172.21.15.39:0/3982446606 learned_addr learned my addr 172.21.15.39:0/3982446606 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff468b9a700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 0x7ff46410dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:29.335 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff469b9c700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4640fe620 0x7ff464114ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 -- 172.21.15.39:0/3982446606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 msgr2=0x7ff46410dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 0x7ff46410dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 -- 172.21.15.39:0/3982446606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4640fe620 msgr2=0x7ff464114ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4640fe620 0x7ff464114ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 -- 172.21.15.39:0/3982446606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600048f0 con 0x7ff4640fca80 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.329+0000 7ff46939b700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 0x7ff46410d710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff460007f80 tx=0x7ff460009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.330+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460004c30 con 0x7ff4640fca80 2023-06-22T18:10:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.330+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c005040 con 0x7ff4640fca80 2023-06-22T18:10:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.330+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff46000f040 con 0x7ff4640fca80 2023-06-22T18:10:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.330+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460012360 con 0x7ff4640fca80 2023-06-22T18:10:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.330+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff454005040 con 0x7ff4640fca80 2023-06-22T18:10:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.331+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4641082c0 con 0x7ff4640fca80 2023-06-22T18:10:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.334+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7ff460013400 con 0x7ff4640fca80 2023-06-22T18:10:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.335+0000 7ff459ffb700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff45009b670 0x7ff45009db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.336+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7ff4600c51f0 con 0x7ff4640fca80 2023-06-22T18:10:29.339 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.336+0000 7ff468b9a700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff45009b670 0x7ff45009db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:29.340 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.336+0000 7ff468b9a700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff45009b670 0x7ff45009db20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff454009b30 tx=0x7ff454009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:29.341 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.337+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff460090ef0 con 0x7ff4640fca80 2023-06-22T18:10:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: Deploying daemon mds.d on smithi039 2023-06-22T18:10:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.39:6834/3433473605,v1:172.21.15.39:6835/3433473605] up:boot 2023-06-22T18:10:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: fsmap 3 up:standby 2023-06-22T18:10:29.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:29 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-22T18:10:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.603+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff464115dd0 con 0x7ff4640fca80 2023-06-22T18:10:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.604+0000 7ff459ffb700 1 -- 172.21.15.39:0/3982446606 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7ff460017090 con 0x7ff4640fca80 2023-06-22T18:10:29.608 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:29.608 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-22T18:10:29.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff45009b670 msgr2=0x7ff45009db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:29.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff45009b670 0x7ff45009db20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff454009b30 tx=0x7ff454009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:29.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 msgr2=0x7ff46410d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:29.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 0x7ff46410d710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff460007f80 tx=0x7ff460009ca0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.610 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 shutdown_connections 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff45009b670 0x7ff45009db20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4640fe620 0x7ff464114ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4640fca80 0x7ff46410d710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 --2- 172.21.15.39:0/3982446606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4640fdc90 0x7ff46410dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 >> 172.21.15.39:0/3982446606 conn(0x7ff4640f81e0 msgr2=0x7ff4640fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 shutdown_connections 2023-06-22T18:10:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:29.606+0000 7ff46a39d700 1 -- 172.21.15.39:0/3982446606 wait complete. 2023-06-22T18:10:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2023-06-22T18:10:29.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:29.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:29.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:29.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: Deploying daemon mds.d on smithi039 2023-06-22T18:10:29.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.39:6834/3433473605,v1:172.21.15.39:6835/3433473605] up:boot 2023-06-22T18:10:29.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: fsmap 3 up:standby 2023-06-22T18:10:29.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:29 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-22T18:10:29.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: Deploying daemon mds.d on smithi039 2023-06-22T18:10:29.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.39:6834/3433473605,v1:172.21.15.39:6835/3433473605] up:boot 2023-06-22T18:10:29.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: fsmap 3 up:standby 2023-06-22T18:10:29.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:29 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-22T18:10:30.610 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:30 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3982446606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:30.610 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:30 smithi039 ceph-mon[109976]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:30.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:30 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3982446606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:30.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:30 smithi133 ceph-mon[116867]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:30.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:30 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3982446606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:30.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:30 smithi101 ceph-mon[116267]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:30.944 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:10:30 smithi039 systemd[1]: Starting Ceph mds.d for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:31.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:31.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/298632238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fe6d0 msgr2=0x7fcbe0104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/298632238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fe6d0 0x7fcbe0104f90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8004ca0 tx=0x7fcbd8009ca0 comp rx=0 tx=0).stop 2023-06-22T18:10:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/298632238 shutdown_connections 2023-06-22T18:10:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/298632238 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fe6d0 0x7fcbe0104f90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/298632238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbe00fcb30 0x7fcbe00fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/298632238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbe00fdd40 0x7fcbe00fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/298632238 >> 172.21.15.39:0/298632238 conn(0x7fcbe00f8290 msgr2=0x7fcbe00fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/298632238 shutdown_connections 2023-06-22T18:10:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.820+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/298632238 wait complete. 2023-06-22T18:10:31.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 Processor -- start 2023-06-22T18:10:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 -- start start 2023-06-22T18:10:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 0x7fcbe010d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbe00fdd40 0x7fcbe010da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbe00fe6d0 0x7fcbe0114af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0115100 con 0x7fcbe00fe6d0 2023-06-22T18:10:31.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0115270 con 0x7fcbe00fcb30 2023-06-22T18:10:31.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe5b9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0115430 con 0x7fcbe00fdd40 2023-06-22T18:10:31.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe4b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 0x7fcbe010d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:31.828 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe4b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 0x7fcbe010d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:37604/0 (socket says 172.21.15.39:37604) 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe4b9b700 1 -- 172.21.15.39:0/487323010 learned_addr learned my addr 172.21.15.39:0/487323010 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.821+0000 7fcbe539c700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbe00fe6d0 0x7fcbe0114af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbdffff700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbe00fdd40 0x7fcbe010da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe4b9b700 1 -- 172.21.15.39:0/487323010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbe00fdd40 msgr2=0x7fcbe010da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe4b9b700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbe00fdd40 0x7fcbe010da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe4b9b700 1 -- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbe00fe6d0 msgr2=0x7fcbe0114af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe4b9b700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbe00fe6d0 0x7fcbe0114af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:31.829 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe4b9b700 1 -- 172.21.15.39:0/487323010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4005040 con 0x7fcbe00fcb30 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe4b9b700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 0x7fcbe010d520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009730 tx=0x7fcbd4009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4007be0 con 0x7fcbe00fcb30 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbd400f040 con 0x7fcbe00fcb30 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4013c30 con 0x7fcbe00fcb30 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd80048f0 con 0x7fcbe00fcb30 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.822+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe01159c0 con 0x7fcbe00fcb30 2023-06-22T18:10:31.830 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.823+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe0108380 con 0x7fcbe00fcb30 2023-06-22T18:10:31.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.825+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fcbd4042bb0 con 0x7fcbe00fcb30 2023-06-22T18:10:31.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.826+0000 7fcbdd7fa700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcbd009b680 0x7fcbd009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:31.831 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.826+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fcbd40c4c60 con 0x7fcbe00fcb30 2023-06-22T18:10:31.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.827+0000 7fcbdffff700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcbd009b680 0x7fcbd009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:31.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.827+0000 7fcbdffff700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcbd009b680 0x7fcbd009db30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009fd0 tx=0x7fcbcc009f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:31.832 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:31.828+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fcbd40908c0 con 0x7fcbe00fcb30 2023-06-22T18:10:31.944 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:10:31 smithi039 bash[152950]: d3c4082745f22e6f943cbde62abed191613319fb159476bec70de00a39e7bca1 2023-06-22T18:10:31.944 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:10:31 smithi039 ceph-mds[153014]: -- 172.21.15.39:0/1753607957 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5572134429c0 con 0x557213418800 2023-06-22T18:10:31.944 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:10:31 smithi039 conmon[152990]: starting mds.d at 2023-06-22T18:10:31.944 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:10:31 smithi039 systemd[1]: Started Ceph mds.d for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.081+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbe0103700 con 0x7fcbe00fcb30 2023-06-22T18:10:32.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.082+0000 7fcbdd7fa700 1 -- 172.21.15.39:0/487323010 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7fcbd4042e60 con 0x7fcbe00fcb30 2023-06-22T18:10:32.086 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:32.087 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-22T18:10:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcbd009b680 msgr2=0x7fcbd009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcbd009b680 0x7fcbd009db30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009fd0 tx=0x7fcbcc009f60 comp rx=0 tx=0).stop 2023-06-22T18:10:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 msgr2=0x7fcbe010d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 0x7fcbe010d520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009730 tx=0x7fcbd4009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 shutdown_connections 2023-06-22T18:10:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fcbd009b680 0x7fcbd009db30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcbe00fe6d0 0x7fcbe0114af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbe00fcb30 0x7fcbe010d520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 --2- 172.21.15.39:0/487323010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbe00fdd40 0x7fcbe010da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 >> 172.21.15.39:0/487323010 conn(0x7fcbe00f8290 msgr2=0x7fcbe00fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 shutdown_connections 2023-06-22T18:10:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:32.085+0000 7fcbe5b9d700 1 -- 172.21.15.39:0/487323010 wait complete. 2023-06-22T18:10:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2023-06-22T18:10:32.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.39:6836/3858396221,v1:172.21.15.39:6837/3858396221] up:boot 2023-06-22T18:10:32.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: fsmap 4 up:standby 2023-06-22T18:10:32.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-22T18:10:32.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:32.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:32.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:32.855 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: Deploying daemon mds.e on smithi101 2023-06-22T18:10:32.856 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:32.856 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:32 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/487323010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.39:6836/3858396221,v1:172.21.15.39:6837/3858396221] up:boot 2023-06-22T18:10:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: fsmap 4 up:standby 2023-06-22T18:10:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-22T18:10:32.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: Deploying daemon mds.e on smithi101 2023-06-22T18:10:32.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:32.946 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:32 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/487323010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.39:6836/3858396221,v1:172.21.15.39:6837/3858396221] up:boot 2023-06-22T18:10:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: fsmap 4 up:standby 2023-06-22T18:10:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-22T18:10:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: Deploying daemon mds.e on smithi101 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:32.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:32 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/487323010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:33.603 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:10:33 smithi101 systemd[1]: Starting Ceph mds.e for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:34.107 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:34.141 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:10:34 smithi101 ceph-mds[147807]: -- 172.21.15.101:0/4137019702 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55764db129c0 con 0x55764dae9000 2023-06-22T18:10:34.141 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:10:34 smithi101 conmon[147782]: starting mds.e at 2023-06-22T18:10:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 -- 172.21.15.39:0/426940500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 msgr2=0x7f27ec104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/426940500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 0x7f27ec104fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27e4004ca0 tx=0x7f27e4009ca0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 -- 172.21.15.39:0/426940500 shutdown_connections 2023-06-22T18:10:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/426940500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 0x7f27ec104fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/426940500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fcac0 0x7f27ec0fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/426940500 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fdd70 0x7f27ec0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 -- 172.21.15.39:0/426940500 >> 172.21.15.39:0/426940500 conn(0x7f27ec0f8240 msgr2=0x7f27ec0fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 -- 172.21.15.39:0/426940500 shutdown_connections 2023-06-22T18:10:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.306+0000 7f27f1b9d700 1 -- 172.21.15.39:0/426940500 wait complete. 2023-06-22T18:10:34.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.307+0000 7f27f1b9d700 1 Processor -- start 2023-06-22T18:10:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.307+0000 7f27f1b9d700 1 -- start start 2023-06-22T18:10:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f1b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 0x7f27ec10d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f1b9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fdd70 0x7f27ec10dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f1b9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 0x7f27ec114b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f1b9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec115180 con 0x7f27ec0fe720 2023-06-22T18:10:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f1b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1152f0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f1b9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1154b0 con 0x7f27ec0fdd70 2023-06-22T18:10:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 0x7f27ec10d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 0x7f27ec10d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:37628/0 (socket says 172.21.15.39:37628) 2023-06-22T18:10:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 -- 172.21.15.39:0/2754323815 learned_addr learned my addr 172.21.15.39:0/2754323815 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f139c700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 0x7f27ec114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27ebfff700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fdd70 0x7f27ec10dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 -- 172.21.15.39:0/2754323815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fdd70 msgr2=0x7f27ec10dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fdd70 0x7f27ec10dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 -- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 msgr2=0x7f27ec114b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 0x7f27ec114b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 -- 172.21.15.39:0/2754323815 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40048f0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.316 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27ebfff700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fdd70 0x7f27ec10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.308+0000 7f27f0b9b700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 0x7f27ec10d5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27e000ebc0 tx=0x7f27e000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.309+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0007cc0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.309+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27e000f040 con 0x7f27ec0fcac0 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.309+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec115730 con 0x7f27ec0fcac0 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.309+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00094a0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.309+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec115cf0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.317 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.310+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec1037f0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.318 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.311+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f27e000c600 con 0x7f27ec0fcac0 2023-06-22T18:10:34.318 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.313+0000 7f27e97fa700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f27dc09b700 0x7f27dc09dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:34.318 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.313+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f27e00c4570 con 0x7f27ec0fcac0 2023-06-22T18:10:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.313+0000 7f27ebfff700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f27dc09b700 0x7f27dc09dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.314+0000 7f27ebfff700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f27dc09b700 0x7f27dc09dbb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27d8007fd0 tx=0x7f27d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:34.320 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.316+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f27e004da00 con 0x7f27ec0fcac0 2023-06-22T18:10:34.573 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.570+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27ec02b060 con 0x7f27ec0fcac0 2023-06-22T18:10:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.570+0000 7f27e97fa700 1 -- 172.21.15.39:0/2754323815 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f27e0019de0 con 0x7f27ec0fcac0 2023-06-22T18:10:34.574 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:34.574 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-22T18:10:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f27dc09b700 msgr2=0x7f27dc09dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f27dc09b700 0x7f27dc09dbb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27d8007fd0 tx=0x7f27d800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 msgr2=0x7f27ec10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 0x7f27ec10d5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27e000ebc0 tx=0x7f27e000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 shutdown_connections 2023-06-22T18:10:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f27dc09b700 0x7f27dc09dbb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0fe720 0x7f27ec114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f27ec0fcac0 0x7f27ec10d5a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 --2- 172.21.15.39:0/2754323815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27ec0fdd70 0x7f27ec10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 >> 172.21.15.39:0/2754323815 conn(0x7f27ec0f8240 msgr2=0x7f27ec0fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 shutdown_connections 2023-06-22T18:10:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:34.573+0000 7f27f1b9d700 1 -- 172.21.15.39:0/2754323815 wait complete. 2023-06-22T18:10:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2023-06-22T18:10:34.602 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:10:34 smithi101 bash[147742]: 8a9d9462a38e67c3890dc322f9a932f9e81fbf5c881ba444a7174ce0b7bc8b86 2023-06-22T18:10:34.603 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:10:34 smithi101 systemd[1]: Started Ceph mds.e for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:35.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:35.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:35.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2754323815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:35.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.101:6834/3939535265,v1:172.21.15.101:6835/3939535265] up:boot 2023-06-22T18:10:35.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: fsmap 5 up:standby 2023-06-22T18:10:35.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:35 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-22T18:10:35.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:35.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:35.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:35.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2754323815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:35.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.101:6834/3939535265,v1:172.21.15.101:6835/3939535265] up:boot 2023-06-22T18:10:35.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: fsmap 5 up:standby 2023-06-22T18:10:35.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:35 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-22T18:10:35.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2754323815' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:35.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.101:6834/3939535265,v1:172.21.15.101:6835/3939535265] up:boot 2023-06-22T18:10:35.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: fsmap 5 up:standby 2023-06-22T18:10:35.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:35 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-22T18:10:36.032 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:10:36 smithi133 systemd[1]: Starting Ceph mds.f for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:36.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:36 smithi133 ceph-mon[116867]: Deploying daemon mds.f on smithi133 2023-06-22T18:10:36.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:36 smithi039 ceph-mon[109976]: Deploying daemon mds.f on smithi133 2023-06-22T18:10:36.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:36.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:36 smithi101 ceph-mon[116267]: Deploying daemon mds.f on smithi133 2023-06-22T18:10:36.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2112190744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94071950 msgr2=0x7f9d9406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:36.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2112190744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94071950 0x7f9d9406fdc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d880049c0 tx=0x7f9d88009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:36.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2112190744 shutdown_connections 2023-06-22T18:10:36.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2112190744 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94104a30 0x7f9d94106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:36.802 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2112190744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94071950 0x7f9d9406fdc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:36.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2112190744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d94070300 0x7f9d94070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:36.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2112190744 >> 172.21.15.39:0/2112190744 conn(0x7f9d940f8290 msgr2=0x7f9d940fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:36.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2112190744 shutdown_connections 2023-06-22T18:10:36.803 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.798+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2112190744 wait complete. 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 Processor -- start 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 -- start start 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 0x7f9d9410f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94071950 0x7f9d9410fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d94104a30 0x7f9d94116f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94117590 con 0x7f9d94104a30 2023-06-22T18:10:36.805 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94117700 con 0x7f9d94071950 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.799+0000 7f9d99b9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941178c0 con 0x7f9d94070300 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 0x7f9d9410f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 0x7f9d9410f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37044/0 (socket says 172.21.15.39:37044) 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 -- 172.21.15.39:0/2890144485 learned_addr learned my addr 172.21.15.39:0/2890144485 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d9939c700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d94104a30 0x7f9d94116f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d93fff700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94071950 0x7f9d9410fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 -- 172.21.15.39:0/2890144485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94071950 msgr2=0x7f9d9410fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94071950 0x7f9d9410fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 -- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d94104a30 msgr2=0x7f9d94116f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d94104a30 0x7f9d94116f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 -- 172.21.15.39:0/2890144485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88005040 con 0x7f9d94070300 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d93fff700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94071950 0x7f9d9410fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d98b9b700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 0x7f9d9410f9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009180 tx=0x7f9d88009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88020070 con 0x7f9d94070300 2023-06-22T18:10:36.807 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d880095d0 con 0x7f9d94070300 2023-06-22T18:10:36.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d94117b40 con 0x7f9d94070300 2023-06-22T18:10:36.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.800+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8800e070 con 0x7f9d94070300 2023-06-22T18:10:36.808 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.801+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d94117fa0 con 0x7f9d94070300 2023-06-22T18:10:36.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.801+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d94103180 con 0x7f9d94070300 2023-06-22T18:10:36.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.803+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f9d8804bbb0 con 0x7f9d94070300 2023-06-22T18:10:36.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.805+0000 7f9d917fa700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f9d8409b700 0x7f9d8409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:36.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.805+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f9d880c5d20 con 0x7f9d94070300 2023-06-22T18:10:36.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.805+0000 7f9d93fff700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f9d8409b700 0x7f9d8409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:36.809 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.805+0000 7f9d93fff700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f9d8409b700 0x7f9d8409dbb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009fd0 tx=0x7f9d8000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:36.810 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:36.807+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f9d88091a00 con 0x7f9d94070300 2023-06-22T18:10:36.983 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:10:36 smithi133 bash[149993]: 88b2098483633de31aef0d726579d042448f019ab0bc27a8828bd28b466ce4d5 2023-06-22T18:10:36.984 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:10:36 smithi133 ceph-mds[150057]: -- 172.21.15.133:0/489612087 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556607e129c0 con 0x556607de8c00 2023-06-22T18:10:36.984 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:10:36 smithi133 conmon[150033]: starting mds.f at 2023-06-22T18:10:36.984 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:10:36 smithi133 systemd[1]: Started Ceph mds.f for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:37.062 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.058+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d9410a570 con 0x7f9d94070300 2023-06-22T18:10:37.063 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.059+0000 7f9d917fa700 1 -- 172.21.15.39:0/2890144485 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4294 (secure 0 0 0) 0x7f9d8804be60 con 0x7f9d94070300 2023-06-22T18:10:37.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:37.064 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-06-22T18:10:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.061+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f9d8409b700 msgr2=0x7f9d8409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.061+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f9d8409b700 0x7f9d8409dbb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009fd0 tx=0x7f9d8000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:37.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 msgr2=0x7f9d9410f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:37.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 0x7f9d9410f9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009180 tx=0x7f9d88009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:37.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 shutdown_connections 2023-06-22T18:10:37.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f9d8409b700 0x7f9d8409dbb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:37.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d94104a30 0x7f9d94116f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:37.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d94070300 0x7f9d9410f9b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:37.068 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 --2- 172.21.15.39:0/2890144485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d94071950 0x7f9d9410fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 >> 172.21.15.39:0/2890144485 conn(0x7f9d940f8290 msgr2=0x7f9d940fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 shutdown_connections 2023-06-22T18:10:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:37.062+0000 7f9d99b9d700 1 -- 172.21.15.39:0/2890144485 wait complete. 2023-06-22T18:10:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 6 2023-06-22T18:10:37.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:37.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.419 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.420 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:37.420 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:37.420 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:37.420 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: Deploying daemon mds.c on smithi133 2023-06-22T18:10:37.420 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:37 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2890144485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:37.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:37.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:37.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: Deploying daemon mds.c on smithi133 2023-06-22T18:10:37.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:37 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2890144485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:37.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:37.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:37.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:37.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:37.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:37.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: Deploying daemon mds.c on smithi133 2023-06-22T18:10:37.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:37 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2890144485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:38.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:38 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.133:6834/412669959,v1:172.21.15.133:6835/412669959] up:boot 2023-06-22T18:10:38.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:38 smithi133 ceph-mon[116867]: fsmap 6 up:standby 2023-06-22T18:10:38.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:38 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-22T18:10:38.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:38 smithi133 ceph-mon[116867]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:38.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:38 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.133:6834/412669959,v1:172.21.15.133:6835/412669959] up:boot 2023-06-22T18:10:38.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:38 smithi039 ceph-mon[109976]: fsmap 6 up:standby 2023-06-22T18:10:38.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:38 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-22T18:10:38.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:38 smithi039 ceph-mon[109976]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:38.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:38 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.133:6834/412669959,v1:172.21.15.133:6835/412669959] up:boot 2023-06-22T18:10:38.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:38 smithi101 ceph-mon[116267]: fsmap 6 up:standby 2023-06-22T18:10:38.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:38 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-22T18:10:38.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:38 smithi101 ceph-mon[116267]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:38.695 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:10:38 smithi133 systemd[1]: Starting Ceph mds.c for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:39.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:39.234 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:10:38 smithi133 ceph-mds[150618]: -- 172.21.15.133:0/3210073305 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9cbfd89c0 con 0x55e9cbfaec00 2023-06-22T18:10:39.234 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:10:38 smithi133 conmon[150594]: starting mds.c at 2023-06-22T18:10:39.234 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:10:38 smithi133 bash[150554]: 48bdeacc851f106486d2e7009e8687f4c3cb8c6ff4294dd7f20c00c9587317cd 2023-06-22T18:10:39.234 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:10:38 smithi133 systemd[1]: Started Ceph mds.c for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/3680070297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 msgr2=0x7f1ef40fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/3680070297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef40fe1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80049c0 tx=0x7f1ee8009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/3680070297 shutdown_connections 2023-06-22T18:10:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/3680070297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ef40fe720 0x7f1ef4104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/3680070297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fcac0 0x7f1ef40fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/3680070297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef40fe1e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/3680070297 >> 172.21.15.39:0/3680070297 conn(0x7f1ef40f8240 msgr2=0x7f1ef40fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/3680070297 shutdown_connections 2023-06-22T18:10:39.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.277+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/3680070297 wait complete. 2023-06-22T18:10:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.278+0000 7f1ef8f9d700 1 Processor -- start 2023-06-22T18:10:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.278+0000 7f1ef8f9d700 1 -- start start 2023-06-22T18:10:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef8f9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ef40fcac0 0x7f1ef4071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef8f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef8f9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fe720 0x7f1ef40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef8f9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4070a30 con 0x7f1ef40fe720 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef8f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4070ba0 con 0x7f1ef40fdd70 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef8f9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef41177f0 con 0x7f1ef40fcac0 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef406fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:58468/0 (socket says 172.21.15.39:58468) 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 -- 172.21.15.39:0/2342894560 learned_addr learned my addr 172.21.15.39:0/2342894560 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:39.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef3fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fe720 0x7f1ef40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef37fe700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ef40fcac0 0x7f1ef4071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 -- 172.21.15.39:0/2342894560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ef40fcac0 msgr2=0x7f1ef4071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ef40fcac0 0x7f1ef4071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 -- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fe720 msgr2=0x7f1ef40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fe720 0x7f1ef40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 -- 172.21.15.39:0/2342894560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8005040 con 0x7f1ef40fdd70 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef3fff700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fe720 0x7f1ef40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.279+0000 7f1ef2ffd700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef406fd80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800b920 tx=0x7f1ee8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.280+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800e040 con 0x7f1ef40fdd70 2023-06-22T18:10:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.280+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef4117a40 con 0x7f1ef40fdd70 2023-06-22T18:10:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.280+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee8009670 con 0x7f1ef40fdd70 2023-06-22T18:10:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.280+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8013c70 con 0x7f1ef40fdd70 2023-06-22T18:10:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.280+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef4117f20 con 0x7f1ef40fdd70 2023-06-22T18:10:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.281+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef4103a20 con 0x7f1ef40fdd70 2023-06-22T18:10:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.283+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f1ee804bbb0 con 0x7f1ef40fdd70 2023-06-22T18:10:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.284+0000 7f1edbfff700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1edc09b680 0x7f1edc09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.284+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f1ee80c5cc0 con 0x7f1ef40fdd70 2023-06-22T18:10:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.284+0000 7f1ef37fe700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1edc09b680 0x7f1edc09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.285+0000 7f1ef37fe700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1edc09b680 0x7f1edc09db30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009b80 tx=0x7f1ee400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.286+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f1ee8091920 con 0x7f1ef40fdd70 2023-06-22T18:10:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.538+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ef402afc0 con 0x7f1ef40fdd70 2023-06-22T18:10:39.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.539+0000 7f1edbfff700 1 -- 172.21.15.39:0/2342894560 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7f1ee804be60 con 0x7f1ef40fdd70 2023-06-22T18:10:39.543 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:39.543 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-22T18:10:39.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1edc09b680 msgr2=0x7f1edc09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1edc09b680 0x7f1edc09db30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009b80 tx=0x7f1ee400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 msgr2=0x7f1ef406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef406fd80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800b920 tx=0x7f1ee8009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 shutdown_connections 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1edc09b680 0x7f1edc09db30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef40fe720 0x7f1ef40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ef40fcac0 0x7f1ef4071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 --2- 172.21.15.39:0/2342894560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ef40fdd70 0x7f1ef406fd80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 >> 172.21.15.39:0/2342894560 conn(0x7f1ef40f8240 msgr2=0x7f1ef40fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 shutdown_connections 2023-06-22T18:10:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:39.542+0000 7f1ef8f9d700 1 -- 172.21.15.39:0/2342894560 wait complete. 2023-06-22T18:10:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2023-06-22T18:10:40.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: Deploying daemon mds.h on smithi101 2023-06-22T18:10:40.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.133:6836/434210144,v1:172.21.15.133:6837/434210144] up:boot 2023-06-22T18:10:40.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: fsmap 7 up:standby 2023-06-22T18:10:40.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-22T18:10:40.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:40 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2342894560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:40.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: Deploying daemon mds.h on smithi101 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.133:6836/434210144,v1:172.21.15.133:6837/434210144] up:boot 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: fsmap 7 up:standby 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-22T18:10:40.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:40 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2342894560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:40.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:40.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:40.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:40.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:40.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: Deploying daemon mds.h on smithi101 2023-06-22T18:10:40.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.133:6836/434210144,v1:172.21.15.133:6837/434210144] up:boot 2023-06-22T18:10:40.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: fsmap 7 up:standby 2023-06-22T18:10:40.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-22T18:10:40.486 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:40 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2342894560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:40.998 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:10:40 smithi101 systemd[1]: Starting Ceph mds.h for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:41.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:41 smithi101 ceph-mon[116267]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:41.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:41 smithi039 ceph-mon[109976]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:41.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:41 smithi133 ceph-mon[116867]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:41.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:41.651 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:10:41 smithi101 ceph-mds[148372]: -- 172.21.15.101:0/2750586534 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55924606c9c0 con 0x559246042c00 2023-06-22T18:10:41.651 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:10:41 smithi101 conmon[148348]: starting mds.h at 2023-06-22T18:10:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.760+0000 7fe37939d700 1 -- 172.21.15.39:0/636637004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 msgr2=0x7fe3740fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.760+0000 7fe37939d700 1 --2- 172.21.15.39:0/636637004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe3740fced0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3640049c0 tx=0x7fe364009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 -- 172.21.15.39:0/636637004 shutdown_connections 2023-06-22T18:10:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 --2- 172.21.15.39:0/636637004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3740fe720 0x7fe374104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 --2- 172.21.15.39:0/636637004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe3740fced0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 --2- 172.21.15.39:0/636637004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3740fdd70 0x7fe3740fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 -- 172.21.15.39:0/636637004 >> 172.21.15.39:0/636637004 conn(0x7fe3740f8240 msgr2=0x7fe3740fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 -- 172.21.15.39:0/636637004 shutdown_connections 2023-06-22T18:10:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 -- 172.21.15.39:0/636637004 wait complete. 2023-06-22T18:10:41.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.761+0000 7fe37939d700 1 Processor -- start 2023-06-22T18:10:41.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 -- start start 2023-06-22T18:10:41.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe37410d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3740fdd70 0x7fe37410dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3740fe720 0x7fe374114cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe374115300 con 0x7fe3740fcac0 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe374115470 con 0x7fe3740fe720 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe37939d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe374115630 con 0x7fe3740fdd70 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe373fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe37410d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:41.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe373fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe37410d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54516/0 (socket says 172.21.15.39:54516) 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe373fff700 1 -- 172.21.15.39:0/1696309482 learned_addr learned my addr 172.21.15.39:0/1696309482 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe378b9c700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3740fe720 0x7fe374114cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.762+0000 7fe3737fe700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3740fdd70 0x7fe37410dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe373fff700 1 -- 172.21.15.39:0/1696309482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3740fdd70 msgr2=0x7fe37410dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe373fff700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3740fdd70 0x7fe37410dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe373fff700 1 -- 172.21.15.39:0/1696309482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3740fe720 msgr2=0x7fe374114cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:41.769 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe373fff700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3740fe720 0x7fe374114cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe373fff700 1 -- 172.21.15.39:0/1696309482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364005040 con 0x7fe3740fcac0 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe373fff700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe37410d720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe36400bfd0 tx=0x7fe364009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36400e040 con 0x7fe3740fcac0 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3741158b0 con 0x7fe3740fcac0 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe364013420 con 0x7fe3740fcac0 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640097a0 con 0x7fe3740fcac0 2023-06-22T18:10:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.763+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe374115d90 con 0x7fe3740fcac0 2023-06-22T18:10:41.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.764+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374103a20 con 0x7fe3740fcac0 2023-06-22T18:10:41.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.766+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fe364013590 con 0x7fe3740fcac0 2023-06-22T18:10:41.771 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.768+0000 7fe370ff9700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe35c09b680 0x7fe35c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:41.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.768+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fe3640c5580 con 0x7fe3740fcac0 2023-06-22T18:10:41.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.768+0000 7fe3737fe700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe35c09b680 0x7fe35c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:41.772 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.769+0000 7fe3737fe700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe35c09b680 0x7fe35c09db30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe368004c40 tx=0x7fe368009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:41.776 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:41.773+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fe364055a00 con 0x7fe3740fcac0 2023-06-22T18:10:42.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.025+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe37402b110 con 0x7fe3740fcac0 2023-06-22T18:10:42.029 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.026+0000 7fe370ff9700 1 -- 172.21.15.39:0/1696309482 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7fe364091280 con 0x7fe3740fcac0 2023-06-22T18:10:42.030 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:42.030 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/4255397239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":4255397239},{"type":"v1","addr":"172.21.15.101:6837","nonce":4255397239}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-22T18:10:42.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe35c09b680 msgr2=0x7fe35c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:42.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe35c09b680 0x7fe35c09db30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe368004c40 tx=0x7fe368009040 comp rx=0 tx=0).stop 2023-06-22T18:10:42.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 msgr2=0x7fe37410d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:42.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe37410d720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe36400bfd0 tx=0x7fe364009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:42.032 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 shutdown_connections 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fe35c09b680 0x7fe35c09db30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3740fe720 0x7fe374114cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3740fcac0 0x7fe37410d720 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 --2- 172.21.15.39:0/1696309482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3740fdd70 0x7fe37410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 >> 172.21.15.39:0/1696309482 conn(0x7fe3740f8240 msgr2=0x7fe3740fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 shutdown_connections 2023-06-22T18:10:42.033 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:42.028+0000 7fe37939d700 1 -- 172.21.15.39:0/1696309482 wait complete. 2023-06-22T18:10:42.034 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 9 2023-06-22T18:10:42.102 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:10:41 smithi101 bash[148308]: ce17ea024e2de403e391d610faa07079ef2b0f6c8aa7930d77cf82ecadabd624 2023-06-22T18:10:42.103 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:10:41 smithi101 systemd[1]: Started Ceph mds.h for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:43.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:43.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: Deploying daemon mds.i on smithi133 2023-06-22T18:10:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.101:6836/4255397239,v1:172.21.15.101:6837/4255397239] up:boot 2023-06-22T18:10:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: fsmap 8 up:standby 2023-06-22T18:10:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-22T18:10:43.104 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:42 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1696309482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: Deploying daemon mds.i on smithi133 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.101:6836/4255397239,v1:172.21.15.101:6837/4255397239] up:boot 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: fsmap 8 up:standby 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-22T18:10:43.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:42 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1696309482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: Deploying daemon mds.i on smithi133 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.101:6836/4255397239,v1:172.21.15.101:6837/4255397239] up:boot 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: fsmap 8 up:standby 2023-06-22T18:10:43.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-22T18:10:43.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:42 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1696309482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:43.878 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:10:43 smithi133 systemd[1]: Starting Ceph mds.i for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:44.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:44.233 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:10:44 smithi133 ceph-mds[151183]: -- 172.21.15.133:0/1985716902 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558eb337e9c0 con 0x558eb3354800 2023-06-22T18:10:44.234 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:10:44 smithi133 conmon[151159]: starting mds.i at 2023-06-22T18:10:44.234 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:10:44 smithi133 bash[151119]: 1bf0b521a0d47e177491bbd566841d5e32202ce80672a249e1b3aa656b7c5f57 2023-06-22T18:10:44.234 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:10:44 smithi133 systemd[1]: Started Ceph mds.i for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:44.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 -- 172.21.15.39:0/4144721791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fcb30 msgr2=0x7f2c240fcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:44.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 --2- 172.21.15.39:0/4144721791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fcb30 0x7f2c240fcf40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c140049c0 tx=0x7f2c14009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:44.255 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 -- 172.21.15.39:0/4144721791 shutdown_connections 2023-06-22T18:10:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 --2- 172.21.15.39:0/4144721791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c240fe6d0 0x7f2c24104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 --2- 172.21.15.39:0/4144721791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fcb30 0x7f2c240fcf40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 --2- 172.21.15.39:0/4144721791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fdd40 0x7f2c240fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 -- 172.21.15.39:0/4144721791 >> 172.21.15.39:0/4144721791 conn(0x7f2c240f8290 msgr2=0x7f2c240fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 -- 172.21.15.39:0/4144721791 shutdown_connections 2023-06-22T18:10:44.256 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.251+0000 7f2c23fff700 1 -- 172.21.15.39:0/4144721791 wait complete. 2023-06-22T18:10:44.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.252+0000 7f2c23fff700 1 Processor -- start 2023-06-22T18:10:44.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.252+0000 7f2c23fff700 1 -- start start 2023-06-22T18:10:44.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c23fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 0x7f2c2410f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:44.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c23fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fdd40 0x7f2c2410fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:44.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c23fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c240fe6d0 0x7f2c24116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:44.258 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c23fff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24117510 con 0x7f2c240fdd40 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c23fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24117680 con 0x7f2c240fcb30 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c23fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c24117840 con 0x7f2c240fe6d0 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c227fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fdd40 0x7f2c2410fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 0x7f2c2410f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 0x7f2c2410f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:58506/0 (socket says 172.21.15.39:58506) 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 -- 172.21.15.39:0/445914281 learned_addr learned my addr 172.21.15.39:0/445914281 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:44.259 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c227fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fdd40 0x7f2c2410fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54534/0 (socket says 172.21.15.39:54534) 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c237fe700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c240fe6d0 0x7f2c24116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 -- 172.21.15.39:0/445914281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c240fe6d0 msgr2=0x7f2c24116f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c240fe6d0 0x7f2c24116f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 -- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fdd40 msgr2=0x7f2c2410fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fdd40 0x7f2c2410fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.253+0000 7f2c22ffd700 1 -- 172.21.15.39:0/445914281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14005040 con 0x7f2c240fcb30 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.254+0000 7f2c22ffd700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 0x7f2c2410f930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c140137e0 tx=0x7f2c14009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:44.260 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.254+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e050 con 0x7f2c240fcb30 2023-06-22T18:10:44.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.254+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c140097b0 con 0x7f2c240fcb30 2023-06-22T18:10:44.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.254+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c24117ac0 con 0x7f2c240fcb30 2023-06-22T18:10:44.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.254+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1401be40 con 0x7f2c240fcb30 2023-06-22T18:10:44.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.254+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c24117fa0 con 0x7f2c240fcb30 2023-06-22T18:10:44.261 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.255+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c24108370 con 0x7f2c240fcb30 2023-06-22T18:10:44.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.258+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f2c1401b4f0 con 0x7f2c240fcb30 2023-06-22T18:10:44.262 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.259+0000 7f2c0b7fe700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2c0c09b680 0x7f2c0c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:44.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.259+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f2c140c6380 con 0x7f2c240fcb30 2023-06-22T18:10:44.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.259+0000 7f2c227fc700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2c0c09b680 0x7f2c0c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:44.263 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.260+0000 7f2c227fc700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2c0c09b680 0x7f2c0c09db30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009b80 tx=0x7f2c1800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:44.264 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.261+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f2c14091fe0 con 0x7f2c240fcb30 2023-06-22T18:10:44.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.515+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c24108500 con 0x7f2c240fcb30 2023-06-22T18:10:44.519 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.516+0000 7f2c0b7fe700 1 -- 172.21.15.39:0/445914281 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7f2c24108500 con 0x7f2c240fcb30 2023-06-22T18:10:44.520 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:44.520 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/4255397239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":4255397239},{"type":"v1","addr":"172.21.15.101:6837","nonce":4255397239}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-22T18:10:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.518+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2c0c09b680 msgr2=0x7f2c0c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.518+0000 7f2c23fff700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2c0c09b680 0x7f2c0c09db30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009b80 tx=0x7f2c1800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.518+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 msgr2=0x7f2c2410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.518+0000 7f2c23fff700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 0x7f2c2410f930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c140137e0 tx=0x7f2c14009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 shutdown_connections 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f2c0c09b680 0x7f2c0c09db30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c240fe6d0 0x7f2c24116f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c240fcb30 0x7f2c2410f930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 --2- 172.21.15.39:0/445914281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c240fdd40 0x7f2c2410fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 >> 172.21.15.39:0/445914281 conn(0x7f2c240f8290 msgr2=0x7f2c24103540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 shutdown_connections 2023-06-22T18:10:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:44.519+0000 7f2c23fff700 1 -- 172.21.15.39:0/445914281 wait complete. 2023-06-22T18:10:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 9 2023-06-22T18:10:45.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:45.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/445914281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:45.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.133:6838/2603125304,v1:172.21.15.133:6839/2603125304] up:boot 2023-06-22T18:10:45.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: fsmap 9 up:standby 2023-06-22T18:10:45.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:45 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-22T18:10:45.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:45.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.392 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.393 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:45.393 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:45.393 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:45.393 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/445914281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:45.394 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.133:6838/2603125304,v1:172.21.15.133:6839/2603125304] up:boot 2023-06-22T18:10:45.394 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: fsmap 9 up:standby 2023-06-22T18:10:45.394 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:45 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-22T18:10:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/445914281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.133:6838/2603125304,v1:172.21.15.133:6839/2603125304] up:boot 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: fsmap 9 up:standby 2023-06-22T18:10:45.485 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:45 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-22T18:10:46.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:46 smithi039 ceph-mon[109976]: Deploying daemon mds.g on smithi039 2023-06-22T18:10:46.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:46 smithi101 ceph-mon[116267]: Deploying daemon mds.g on smithi039 2023-06-22T18:10:46.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:46 smithi133 ceph-mon[116867]: Deploying daemon mds.g on smithi039 2023-06-22T18:10:46.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:46.759 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.755+0000 7f5cff59e700 1 -- 172.21.15.39:0/2929526974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fcb10 msgr2=0x7f5d000fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:46.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.755+0000 7f5cff59e700 1 --2- 172.21.15.39:0/2929526974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fcb10 0x7f5d000fcf20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00049c0 tx=0x7f5cf0009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:46.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.755+0000 7f5cff59e700 1 -- 172.21.15.39:0/2929526974 shutdown_connections 2023-06-22T18:10:46.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.755+0000 7f5cff59e700 1 --2- 172.21.15.39:0/2929526974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 0x7f5d00104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:46.760 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.755+0000 7f5cff59e700 1 --2- 172.21.15.39:0/2929526974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fcb10 0x7f5d000fcf20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:46.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.755+0000 7f5cff59e700 1 --2- 172.21.15.39:0/2929526974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fdd20 0x7f5d000fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:46.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.756+0000 7f5cff59e700 1 -- 172.21.15.39:0/2929526974 >> 172.21.15.39:0/2929526974 conn(0x7f5d000f8270 msgr2=0x7f5d000fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:46.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.756+0000 7f5cff59e700 1 -- 172.21.15.39:0/2929526974 shutdown_connections 2023-06-22T18:10:46.761 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.756+0000 7f5cff59e700 1 -- 172.21.15.39:0/2929526974 wait complete. 2023-06-22T18:10:46.762 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.756+0000 7f5cff59e700 1 Processor -- start 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 -- start start 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fcb10 0x7f5d0010fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 0x7f5d0010ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 0x7f5d00116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001175e0 con 0x7f5d000fdd20 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00117750 con 0x7f5d000fe6b0 2023-06-22T18:10:46.763 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cff59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00117910 con 0x7f5d000fcb10 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cfdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 0x7f5d0010ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cfdd9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 0x7f5d0010ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58670/0 (socket says 172.21.15.39:58670) 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cfe59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fcb10 0x7f5d0010fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cfdd9b700 1 -- 172.21.15.39:0/4249066069 learned_addr learned my addr 172.21.15.39:0/4249066069 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.757+0000 7f5cfed9d700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 0x7f5d00116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfdd9b700 1 -- 172.21.15.39:0/4249066069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fcb10 msgr2=0x7f5d0010fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:46.764 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfdd9b700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fcb10 0x7f5d0010fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfdd9b700 1 -- 172.21.15.39:0/4249066069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 msgr2=0x7f5d00116fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfdd9b700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 0x7f5d00116fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfdd9b700 1 -- 172.21.15.39:0/4249066069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0005040 con 0x7f5d000fdd20 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfed9d700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 0x7f5d00116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfe59c700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fcb10 0x7f5d0010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cfdd9b700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 0x7f5d0010ff40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ebc0 tx=0x7f5cf400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4007cc0 con 0x7f5d000fdd20 2023-06-22T18:10:46.765 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cf400f040 con 0x7f5d000fdd20 2023-06-22T18:10:46.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d00117b90 con 0x7f5d000fdd20 2023-06-22T18:10:46.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40094a0 con 0x7f5d000fdd20 2023-06-22T18:10:46.766 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.758+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d001180d0 con 0x7f5d000fdd20 2023-06-22T18:10:46.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.760+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0010a370 con 0x7f5d000fdd20 2023-06-22T18:10:46.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.761+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f5cf400c600 con 0x7f5d000fdd20 2023-06-22T18:10:46.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.763+0000 7f5ceeffd700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5ce809b700 0x7f5ce809dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:46.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.763+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f5cf40c4560 con 0x7f5d000fdd20 2023-06-22T18:10:46.767 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.763+0000 7f5cfe59c700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5ce809b700 0x7f5ce809dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:46.768 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.764+0000 7f5cfe59c700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5ce809b700 0x7f5ce809dbb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0004890 tx=0x7f5cf0009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:46.774 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:46.772+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f5cf404da00 con 0x7f5d000fdd20 2023-06-22T18:10:46.944 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:10:46 smithi039 systemd[1]: Starting Ceph mds.g for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:47.055 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.053+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d00108350 con 0x7f5d000fdd20 2023-06-22T18:10:47.057 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.053+0000 7f5ceeffd700 1 -- 172.21.15.39:0/4249066069 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7330 (secure 0 0 0) 0x7f5cf40901c0 con 0x7f5d000fdd20 2023-06-22T18:10:47.057 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:47.057 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/4255397239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":4255397239},{"type":"v1","addr":"172.21.15.101:6837","nonce":4255397239}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2603125304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2603125304},{"type":"v1","addr":"172.21.15.133:6839","nonce":2603125304}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-22T18:10:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5ce809b700 msgr2=0x7f5ce809dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5ce809b700 0x7f5ce809dbb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0004890 tx=0x7f5cf0009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 msgr2=0x7f5d0010ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 0x7f5d0010ff40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ebc0 tx=0x7f5cf400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 shutdown_connections 2023-06-22T18:10:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f5ce809b700 0x7f5ce809dbb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5d000fe6b0 0x7f5d00116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d000fcb10 0x7f5d0010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 --2- 172.21.15.39:0/4249066069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000fdd20 0x7f5d0010ff40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 >> 172.21.15.39:0/4249066069 conn(0x7f5d000f8270 msgr2=0x7f5d00103600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 shutdown_connections 2023-06-22T18:10:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:47.056+0000 7f5cff59e700 1 -- 172.21.15.39:0/4249066069 wait complete. 2023-06-22T18:10:47.062 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2023-06-22T18:10:47.332 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:47 smithi039 ceph-mon[109976]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:47.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:47 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/4249066069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:47.352 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:47 smithi101 ceph-mon[116267]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:47.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:47 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/4249066069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:47.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:47 smithi133 ceph-mon[116867]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:47.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:47 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/4249066069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:47.694 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:10:47 smithi039 bash[153807]: 7efd6c1c4b4f166b7467ff5e1af1be4700334c450d04af82574cc159b78a7015 2023-06-22T18:10:47.694 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:10:47 smithi039 ceph-mds[153881]: -- 172.21.15.39:0/1632202202 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ceef6509c0 con 0x55ceef627000 2023-06-22T18:10:47.695 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:10:47 smithi039 conmon[153857]: starting mds.g at 2023-06-22T18:10:47.695 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:10:47 smithi039 systemd[1]: Started Ceph mds.g for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:48.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:48.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:48.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:48.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: Deploying daemon mds.a on smithi039 2023-06-22T18:10:48.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:48.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.39:6838/897635640,v1:172.21.15.39:6839/897635640] up:boot 2023-06-22T18:10:48.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: fsmap 10 up:standby 2023-06-22T18:10:48.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:48 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-22T18:10:48.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.904 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: Deploying daemon mds.a on smithi039 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:48.905 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.39:6838/897635640,v1:172.21.15.39:6839/897635640] up:boot 2023-06-22T18:10:48.906 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: fsmap 10 up:standby 2023-06-22T18:10:48.906 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:48 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-22T18:10:48.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:48.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:48.985 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:48.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:48.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: Deploying daemon mds.a on smithi039 2023-06-22T18:10:48.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:48.986 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.39:6838/897635640,v1:172.21.15.39:6839/897635640] up:boot 2023-06-22T18:10:48.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: fsmap 10 up:standby 2023-06-22T18:10:48.987 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:48 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-22T18:10:49.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:49.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 -- 172.21.15.39:0/4245533426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 msgr2=0x7f84640fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:49.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 --2- 172.21.15.39:0/4245533426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f84640fcf20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f84540049c0 tx=0x7f8454009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:49.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 -- 172.21.15.39:0/4245533426 shutdown_connections 2023-06-22T18:10:49.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 --2- 172.21.15.39:0/4245533426 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 0x7f8464104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 --2- 172.21.15.39:0/4245533426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f84640fcf20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 --2- 172.21.15.39:0/4245533426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84640fdd20 0x7f84640fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 -- 172.21.15.39:0/4245533426 >> 172.21.15.39:0/4245533426 conn(0x7f84640f8270 msgr2=0x7f84640fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:49.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 -- 172.21.15.39:0/4245533426 shutdown_connections 2023-06-22T18:10:49.295 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.291+0000 7f8468d9d700 1 -- 172.21.15.39:0/4245533426 wait complete. 2023-06-22T18:10:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.292+0000 7f8468d9d700 1 Processor -- start 2023-06-22T18:10:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.292+0000 7f8468d9d700 1 -- start start 2023-06-22T18:10:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8468d9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f8464071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8468d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84640fdd20 0x7f846406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8468d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 0x7f84640702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8468d9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464070a30 con 0x7f84640fcb10 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8468d9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464070ba0 con 0x7f84640fe6b0 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8468d9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464117900 con 0x7f84640fdd20 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f8464071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8462ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84640fdd20 0x7f846406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f8464071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58698/0 (socket says 172.21.15.39:58698) 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8463fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 0x7f84640702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 -- 172.21.15.39:0/376345203 learned_addr learned my addr 172.21.15.39:0/376345203 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 -- 172.21.15.39:0/376345203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84640fdd20 msgr2=0x7f846406fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84640fdd20 0x7f846406fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 -- 172.21.15.39:0/376345203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 msgr2=0x7f84640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 0x7f84640702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f84637fe700 1 -- 172.21.15.39:0/376345203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454005040 con 0x7f84640fcb10 2023-06-22T18:10:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.293+0000 7f8463fff700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 0x7f84640702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.294+0000 7f84637fe700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f8464071720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f845400b8f0 tx=0x7f8454009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.294+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400e040 con 0x7f84640fcb10 2023-06-22T18:10:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.294+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8464117b50 con 0x7f84640fcb10 2023-06-22T18:10:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.294+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8454013420 con 0x7f84640fcb10 2023-06-22T18:10:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.294+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845401c660 con 0x7f84640fcb10 2023-06-22T18:10:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.294+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8464117fb0 con 0x7f84640fcb10 2023-06-22T18:10:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.295+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8464103ff0 con 0x7f84640fcb10 2023-06-22T18:10:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.297+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f8454013800 con 0x7f84640fcb10 2023-06-22T18:10:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.298+0000 7f844bfff700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f844c09b680 0x7f844c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.298+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f84540c5a70 con 0x7f84640fcb10 2023-06-22T18:10:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.298+0000 7f8462ffd700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f844c09b680 0x7f844c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.299+0000 7f8462ffd700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f844c09b680 0x7f844c09db30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8458004f40 tx=0x7f845800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.306+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f84540916d0 con 0x7f84640fcb10 2023-06-22T18:10:49.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.584+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8464108300 con 0x7f84640fcb10 2023-06-22T18:10:49.587 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.585+0000 7f844bfff700 1 -- 172.21.15.39:0/376345203 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8085 (secure 0 0 0) 0x7f8454094cf0 con 0x7f84640fcb10 2023-06-22T18:10:49.588 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:49.588 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/4255397239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":4255397239},{"type":"v1","addr":"172.21.15.101:6837","nonce":4255397239}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/897635640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":897635640},{"type":"v1","addr":"172.21.15.39:6839","nonce":897635640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2603125304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2603125304},{"type":"v1","addr":"172.21.15.133:6839","nonce":2603125304}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-22T18:10:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.587+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f844c09b680 msgr2=0x7f844c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.587+0000 7f8468d9d700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f844c09b680 0x7f844c09db30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8458004f40 tx=0x7f845800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:49.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.587+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 msgr2=0x7f8464071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:49.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.587+0000 7f8468d9d700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f8464071720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f845400b8f0 tx=0x7f8454009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:49.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 shutdown_connections 2023-06-22T18:10:49.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f844c09b680 0x7f844c09db30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84640fe6b0 0x7f84640702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.591 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640fcb10 0x7f8464071720 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 --2- 172.21.15.39:0/376345203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84640fdd20 0x7f846406fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:49.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 >> 172.21.15.39:0/376345203 conn(0x7f84640f8270 msgr2=0x7f8464101170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:49.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 shutdown_connections 2023-06-22T18:10:49.592 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:49.588+0000 7f8468d9d700 1 -- 172.21.15.39:0/376345203 wait complete. 2023-06-22T18:10:49.592 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 11 2023-06-22T18:10:49.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:49 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/376345203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:49.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:49 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/376345203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:50.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:49 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/376345203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:50.194 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:10:49 smithi039 systemd[1]: Starting Ceph mds.a for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:50.944 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:50.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:50.945 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:10:50 smithi039 bash[154428]: 6317170c10c61b3ad28e0399e0a9abc647e4c485ecf40372f2f25e1bcf7d8a30 2023-06-22T18:10:50.945 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:10:50 smithi039 ceph-mds[154492]: -- 172.21.15.39:0/3751781423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56146aaaa9c0 con 0x56146aa80c00 2023-06-22T18:10:50.946 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:10:50 smithi039 conmon[154468]: starting mds.a at 2023-06-22T18:10:50.946 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:10:50 smithi039 systemd[1]: Started Ceph mds.a for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:51.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:51.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:50 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-22T18:10:51.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-22T18:10:51.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:50 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:51.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:51.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 -- 172.21.15.39:0/3453280666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fe6b0 msgr2=0x7f7858104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:51.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3453280666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fe6b0 0x7f7858104f70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7850009d60 tx=0x7f7850004d10 comp rx=0 tx=0).stop 2023-06-22T18:10:51.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 -- 172.21.15.39:0/3453280666 shutdown_connections 2023-06-22T18:10:51.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3453280666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fe6b0 0x7f7858104f70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:51.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3453280666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f78580fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:51.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3453280666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78580fdd20 0x7f78580fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:51.819 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.814+0000 7f785eb9d700 1 -- 172.21.15.39:0/3453280666 >> 172.21.15.39:0/3453280666 conn(0x7f78580f8270 msgr2=0x7f78580fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:51.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.815+0000 7f785eb9d700 1 -- 172.21.15.39:0/3453280666 shutdown_connections 2023-06-22T18:10:51.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.815+0000 7f785eb9d700 1 -- 172.21.15.39:0/3453280666 wait complete. 2023-06-22T18:10:51.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.815+0000 7f785eb9d700 1 Processor -- start 2023-06-22T18:10:51.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.815+0000 7f785eb9d700 1 -- start start 2023-06-22T18:10:51.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785eb9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f785810d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:51.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785eb9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fdd20 0x7f785810daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785eb9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78580fe6b0 0x7f7858114b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785eb9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858115190 con 0x7f78580fcb10 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785eb9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858115300 con 0x7f78580fdd20 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785eb9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78581154c0 con 0x7f78580fe6b0 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f785810d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785d39a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fdd20 0x7f785810daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f785810d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58730/0 (socket says 172.21.15.39:58730) 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785d39a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fdd20 0x7f785810daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:49582/0 (socket says 172.21.15.39:49582) 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 -- 172.21.15.39:0/3378499094 learned_addr learned my addr 172.21.15.39:0/3378499094 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785e39c700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78580fe6b0 0x7f7858114b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 -- 172.21.15.39:0/3378499094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78580fe6b0 msgr2=0x7f7858114b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78580fe6b0 0x7f7858114b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 -- 172.21.15.39:0/3378499094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fdd20 msgr2=0x7f785810daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:51.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fdd20 0x7f785810daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.816+0000 7f785db9b700 1 -- 172.21.15.39:0/3378499094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854005040 con 0x7f78580fcb10 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.817+0000 7f785db9b700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f785810d5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7854009750 tx=0x7f7854009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.817+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785400e5d0 con 0x7f78580fcb10 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.817+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7850009940 con 0x7f78580fcb10 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.817+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785400f040 con 0x7f78580fcb10 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.817+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7854013ad0 con 0x7f78580fcb10 2023-06-22T18:10:51.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.817+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7858115b70 con 0x7f78580fcb10 2023-06-22T18:10:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.818+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7858108360 con 0x7f78580fcb10 2023-06-22T18:10:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.820+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f785400e730 con 0x7f78580fcb10 2023-06-22T18:10:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.821+0000 7f784e7fc700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f784409b700 0x7f784409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.821+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f78540c5300 con 0x7f78580fcb10 2023-06-22T18:10:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.822+0000 7f785d39a700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f784409b700 0x7f784409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.822+0000 7f785d39a700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f784409b700 0x7f784409dbb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7848009b80 tx=0x7f784800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:51.823+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f7854090f60 con 0x7f78580fcb10 2023-06-22T18:10:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:51 smithi039 ceph-mon[109976]: Deploying daemon mds.b on smithi101 2023-06-22T18:10:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:51 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.39:6840/3954741659,v1:172.21.15.39:6841/3954741659] up:boot 2023-06-22T18:10:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:51 smithi039 ceph-mon[109976]: fsmap 11 up:standby 2023-06-22T18:10:51.945 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:51 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-22T18:10:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.080+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7858115e50 con 0x7f78580fcb10 2023-06-22T18:10:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.080+0000 7f784e7fc700 1 -- 172.21.15.39:0/3378499094 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8843 (secure 0 0 0) 0x7f7854094580 con 0x7f78580fcb10 2023-06-22T18:10:52.084 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:52.084 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/3954741659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3954741659},{"type":"v1","addr":"172.21.15.39:6841","nonce":3954741659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/4255397239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":4255397239},{"type":"v1","addr":"172.21.15.101:6837","nonce":4255397239}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/897635640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":897635640},{"type":"v1","addr":"172.21.15.39:6839","nonce":897635640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2603125304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2603125304},{"type":"v1","addr":"172.21.15.133:6839","nonce":2603125304}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-22T18:10:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.082+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f784409b700 msgr2=0x7f784409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.082+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f784409b700 0x7f784409dbb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7848009b80 tx=0x7f784800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 msgr2=0x7f785810d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f785810d5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7854009750 tx=0x7f7854009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 shutdown_connections 2023-06-22T18:10:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f784409b700 0x7f784409dbb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78580fe6b0 0x7f7858114b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580fcb10 0x7f785810d5b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 --2- 172.21.15.39:0/3378499094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f78580fdd20 0x7f785810daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 >> 172.21.15.39:0/3378499094 conn(0x7f78580f8270 msgr2=0x7f78580fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 shutdown_connections 2023-06-22T18:10:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:52.083+0000 7f785eb9d700 1 -- 172.21.15.39:0/3378499094 wait complete. 2023-06-22T18:10:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 12 2023-06-22T18:10:52.102 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:51 smithi101 ceph-mon[116267]: Deploying daemon mds.b on smithi101 2023-06-22T18:10:52.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:51 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.39:6840/3954741659,v1:172.21.15.39:6841/3954741659] up:boot 2023-06-22T18:10:52.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:51 smithi101 ceph-mon[116267]: fsmap 11 up:standby 2023-06-22T18:10:52.103 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:51 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-22T18:10:52.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:51 smithi133 ceph-mon[116867]: Deploying daemon mds.b on smithi101 2023-06-22T18:10:52.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:51 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.39:6840/3954741659,v1:172.21.15.39:6841/3954741659] up:boot 2023-06-22T18:10:52.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:51 smithi133 ceph-mon[116867]: fsmap 11 up:standby 2023-06-22T18:10:52.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:51 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-22T18:10:53.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:52 smithi101 ceph-mon[116267]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:53.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:52 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3378499094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:53.073 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:10:52 smithi101 systemd[1]: Starting Ceph mds.b for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:10:53.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:52 smithi039 ceph-mon[109976]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:53.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:52 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3378499094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:52 smithi133 ceph-mon[116867]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:53.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:52 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3378499094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:53.602 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:10:53 smithi101 ceph-mds[148937]: -- 172.21.15.101:0/2714034864 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dfd662c9c0 con 0x55dfd6603000 2023-06-22T18:10:53.603 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:10:53 smithi101 conmon[148913]: starting mds.b at 2023-06-22T18:10:53.603 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:10:53 smithi101 bash[148873]: b32d366597b1d70eba4f472ba346a7a1e9b6ef9a1f9550767cb3e380e6216931 2023-06-22T18:10:53.603 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:10:53 smithi101 systemd[1]: Started Ceph mds.b for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:10:54.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-22T18:10:54.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 -- 172.21.15.39:0/998102733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fcac0 msgr2=0x7fad7c0fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 --2- 172.21.15.39:0/998102733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fcac0 0x7fad7c0fced0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0049c0 tx=0x7fad6c009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:54.306 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 -- 172.21.15.39:0/998102733 shutdown_connections 2023-06-22T18:10:54.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 --2- 172.21.15.39:0/998102733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad7c0fe720 0x7fad7c104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 --2- 172.21.15.39:0/998102733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fcac0 0x7fad7c0fced0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 --2- 172.21.15.39:0/998102733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fdd70 0x7fad7c0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 -- 172.21.15.39:0/998102733 >> 172.21.15.39:0/998102733 conn(0x7fad7c0f8240 msgr2=0x7fad7c0fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:54.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 -- 172.21.15.39:0/998102733 shutdown_connections 2023-06-22T18:10:54.307 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.303+0000 7fad80b9d700 1 -- 172.21.15.39:0/998102733 wait complete. 2023-06-22T18:10:54.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.304+0000 7fad80b9d700 1 Processor -- start 2023-06-22T18:10:54.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.304+0000 7fad80b9d700 1 -- start start 2023-06-22T18:10:54.309 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad80b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 0x7fad7c10f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad80b9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fdd70 0x7fad7c10feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad80b9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad7c0fe720 0x7fad7c116f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad80b9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c117550 con 0x7fad7c0fe720 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad80b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c1176c0 con 0x7fad7c0fcac0 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad80b9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c117880 con 0x7fad7c0fdd70 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 0x7fad7c10f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 0x7fad7c10f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:49606/0 (socket says 172.21.15.39:49606) 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 -- 172.21.15.39:0/3379567419 learned_addr learned my addr 172.21.15.39:0/3379567419 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7affd700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fdd70 0x7fad7c10feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7bfff700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad7c0fe720 0x7fad7c116f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 -- 172.21.15.39:0/3379567419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fdd70 msgr2=0x7fad7c10feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fdd70 0x7fad7c10feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 -- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad7c0fe720 msgr2=0x7fad7c116f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad7c0fe720 0x7fad7c116f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 -- 172.21.15.39:0/3379567419 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c005040 con 0x7fad7c0fcac0 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7affd700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fdd70 0x7fad7c10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.305+0000 7fad7b7fe700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 0x7fad7c10f970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b0a0 tx=0x7fad6c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.306+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00e050 con 0x7fad7c0fcac0 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.306+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c117b00 con 0x7fad7c0fcac0 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.306+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad6c01b3f0 con 0x7fad7c0fcac0 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.306+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c012630 con 0x7fad7c0fcac0 2023-06-22T18:10:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.306+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c117fb0 con 0x7fad7c0fcac0 2023-06-22T18:10:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.307+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c108320 con 0x7fad7c0fcac0 2023-06-22T18:10:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.309+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fad6c01b7f0 con 0x7fad7c0fcac0 2023-06-22T18:10:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.311+0000 7fad63fff700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fad6409b680 0x7fad6409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.311+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fad6c0c64d0 con 0x7fad7c0fcac0 2023-06-22T18:10:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.311+0000 7fad7affd700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fad6409b680 0x7fad6409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.312+0000 7fad7affd700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fad6409b680 0x7fad6409db30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b80 tx=0x7fad7000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:54.315 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.313+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fad6c092130 con 0x7fad7c0fcac0 2023-06-22T18:10:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.576+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad7c1181a0 con 0x7fad7c0fcac0 2023-06-22T18:10:54.580 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.577+0000 7fad63fff700 1 -- 172.21.15.39:0/3379567419 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9604 (secure 0 0 0) 0x7fad6c095750 con 0x7fad7c0fcac0 2023-06-22T18:10:54.581 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:54.581 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3433473605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3433473605},{"type":"v1","addr":"172.21.15.39:6835","nonce":3433473605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14460,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3858396221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3858396221},{"type":"v1","addr":"172.21.15.39:6837","nonce":3858396221}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/3954741659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":3954741659},{"type":"v1","addr":"172.21.15.39:6841","nonce":3954741659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/4229656332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":4229656332},{"type":"v1","addr":"172.21.15.101:6833","nonce":4229656332}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2685285663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2685285663},{"type":"v1","addr":"172.21.15.133:6833","nonce":2685285663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/3939535265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":3939535265},{"type":"v1","addr":"172.21.15.101:6835","nonce":3939535265}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/4255397239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":4255397239},{"type":"v1","addr":"172.21.15.101:6837","nonce":4255397239}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/412669959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":412669959},{"type":"v1","addr":"172.21.15.133:6835","nonce":412669959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24515,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/897635640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":897635640},{"type":"v1","addr":"172.21.15.39:6839","nonce":897635640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/434210144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":434210144},{"type":"v1","addr":"172.21.15.133:6837","nonce":434210144}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2603125304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2603125304},{"type":"v1","addr":"172.21.15.133:6839","nonce":2603125304}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24539,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/3279622767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":3279622767},{"type":"v1","addr":"172.21.15.101:6839","nonce":3279622767}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-06-22T18:10:54.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.579+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fad6409b680 msgr2=0x7fad6409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.582 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.579+0000 7fad80b9d700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fad6409b680 0x7fad6409db30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad70009b80 tx=0x7fad7000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.579+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 msgr2=0x7fad7c10f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.579+0000 7fad80b9d700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 0x7fad7c10f970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b0a0 tx=0x7fad6c009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 shutdown_connections 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fad6409b680 0x7fad6409db30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad7c0fe720 0x7fad7c116f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad7c0fcac0 0x7fad7c10f970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 --2- 172.21.15.39:0/3379567419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0fdd70 0x7fad7c10feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.583 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 >> 172.21.15.39:0/3379567419 conn(0x7fad7c0f8240 msgr2=0x7fad7c0fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:54.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 shutdown_connections 2023-06-22T18:10:54.584 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.580+0000 7fad80b9d700 1 -- 172.21.15.39:0/3379567419 wait complete. 2023-06-22T18:10:54.584 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2023-06-22T18:10:54.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T18:10:54.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:54.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:54.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.101:6838/3279622767,v1:172.21.15.101:6839/3279622767] up:boot 2023-06-22T18:10:54.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: fsmap 12 up:standby 2023-06-22T18:10:54.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:54 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-22T18:10:54.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:54.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:54.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.101:6838/3279622767,v1:172.21.15.101:6839/3279622767] up:boot 2023-06-22T18:10:54.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: fsmap 12 up:standby 2023-06-22T18:10:54.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:54 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-22T18:10:54.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 -- 172.21.15.39:0/2452833955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2400fed40 msgr2=0x7ff2400ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 --2- 172.21.15.39:0/2452833955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2400fed40 0x7ff2400ff150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff230004900 tx=0x7ff230009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:54.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 -- 172.21.15.39:0/2452833955 shutdown_connections 2023-06-22T18:10:54.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 --2- 172.21.15.39:0/2452833955 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2401008e0 0x7ff2401071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 --2- 172.21.15.39:0/2452833955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2400fed40 0x7ff2400ff150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 --2- 172.21.15.39:0/2452833955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2400fff50 0x7ff2401003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 -- 172.21.15.39:0/2452833955 >> 172.21.15.39:0/2452833955 conn(0x7ff2400fa580 msgr2=0x7ff2400fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 -- 172.21.15.39:0/2452833955 shutdown_connections 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.813+0000 7ff23ffff700 1 -- 172.21.15.39:0/2452833955 wait complete. 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.814+0000 7ff23ffff700 1 Processor -- start 2023-06-22T18:10:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.814+0000 7ff23ffff700 1 -- start start 2023-06-22T18:10:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 0x7ff24010fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2400fff50 0x7ff24010ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.820 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2401008e0 0x7ff240116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23ffff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401175e0 con 0x7ff2400fff50 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23ffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240117750 con 0x7ff2400fed40 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23ffff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240117910 con 0x7ff2401008e0 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 0x7ff24010fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 0x7ff24010fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:49624/0 (socket says 172.21.15.39:49624) 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 -- 172.21.15.39:0/3370379771 learned_addr learned my addr 172.21.15.39:0/3370379771 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23e7fc700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2400fff50 0x7ff24010ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.821 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23f7fe700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2401008e0 0x7ff240116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 -- 172.21.15.39:0/3370379771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2401008e0 msgr2=0x7ff240116fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2401008e0 0x7ff240116fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 -- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2400fff50 msgr2=0x7ff24010ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2400fff50 0x7ff24010ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 -- 172.21.15.39:0/3370379771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234005040 con 0x7ff2400fed40 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.815+0000 7ff23effd700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 0x7ff24010fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff230004f70 tx=0x7ff230009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.816+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000e040 con 0x7ff2400fed40 2023-06-22T18:10:54.822 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.816+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff230000c90 con 0x7ff2400fed40 2023-06-22T18:10:54.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.816+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23001ce10 con 0x7ff2400fed40 2023-06-22T18:10:54.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.816+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff230005040 con 0x7ff2400fed40 2023-06-22T18:10:54.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.816+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff240117e30 con 0x7ff2400fed40 2023-06-22T18:10:54.823 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.817+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff240106230 con 0x7ff2400fed40 2023-06-22T18:10:54.824 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.818+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7ff230009630 con 0x7ff2400fed40 2023-06-22T18:10:54.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.820+0000 7ff2277fe700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff22809b680 0x7ff22809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:54.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.820+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7ff2300c5ee0 con 0x7ff2400fed40 2023-06-22T18:10:54.825 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.820+0000 7ff23e7fc700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff22809b680 0x7ff22809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:54.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.820+0000 7ff23e7fc700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff22809b680 0x7ff22809db30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff234009fd0 tx=0x7ff23400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:54.826 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:54.824+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7ff230091b40 con 0x7ff2400fed40 2023-06-22T18:10:54.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:54.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.101:6838/3279622767,v1:172.21.15.101:6839/3279622767] up:boot 2023-06-22T18:10:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: fsmap 12 up:standby 2023-06-22T18:10:54.854 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:54 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-22T18:10:55.037 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.035+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff24002b1e0 con 0x7ff2400fed40 2023-06-22T18:10:55.038 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.036+0000 7ff2277fe700 1 -- 172.21.15.39:0/3370379771 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18120 (secure 0 0 0) 0x7ff230095160 con 0x7ff2400fed40 2023-06-22T18:10:55.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:55.039 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","created":"2023-06-22T18:02:53.160326+0000","modified":"2023-06-22T18:10:12.224478+0000","last_up_change":"2023-06-22T18:10:10.873966+0000","last_in_change":"2023-06-22T18:09:51.312474+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T18:06:20.190713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"f55d12a1-86b5-42a6-ad35-d5eaf3d99c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6803","nonce":471291770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6805","nonce":471291770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6809","nonce":471291770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6807","nonce":471291770}]},"public_addr":"172.21.15.39:6803/471291770","cluster_addr":"172.21.15.39:6805/471291770","heartbeat_back_addr":"172.21.15.39:6809/471291770","heartbeat_front_addr":"172.21.15.39:6807/471291770","state":["exists","up"]},{"osd":1,"uuid":"4d8fb427-c4b2-45c3-ac42-dea08c42e4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6801","nonce":840111728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6803","nonce":840111728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6807","nonce":840111728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6805","nonce":840111728}]},"public_addr":"172.21.15.101:6801/840111728","cluster_addr":"172.21.15.101:6803/840111728","heartbeat_back_addr":"172.21.15.101:6807/840111728","heartbeat_front_addr":"172.21.15.101:6805/840111728","state":["exists","up"]},{"osd":2,"uuid":"9c714cec-14fc-4aed-b3f1-f0773edd197c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6801","nonce":3165338416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6803","nonce":3165338416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6807","nonce":3165338416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6805","nonce":3165338416}]},"public_addr":"172.21.15.133:6801/3165338416","cluster_addr":"172.21.15.133:6803/3165338416","heartbeat_back_addr":"172.21.15.133:6807/3165338416","heartbeat_front_addr":"172.21.15.133:6805/3165338416","state":["exists","up"]},{"osd":3,"uuid":"71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6811","nonce":3695104438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6813","nonce":3695104438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6817","nonce":3695104438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6815","nonce":3695104438}]},"public_addr":"172.21.15.39:6811/3695104438","cluster_addr":"172.21.15.39:6813/3695104438","heartbeat_back_addr":"172.21.15.39:6817/3695104438","heartbeat_front_addr":"172.21.15.39:6815/3695104438","state":["exists","up"]},{"osd":4,"uuid":"c9ecace5-1032-4f25-a5f3-84ec819c83ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6809","nonce":2726745315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6811","nonce":2726745315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6815","nonce":2726745315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6813","nonce":2726745315}]},"public_addr":"172.21.15.101:6809/2726745315","cluster_addr":"172.21.15.101:6811/2726745315","heartbeat_back_addr":"172.21.15.101:6815/2726745315","heartbeat_front_addr":"172.21.15.101:6813/2726745315","state":["exists","up"]},{"osd":5,"uuid":"379ed499-68d9-4032-bafd-31cb02995388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6809","nonce":1467210441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6811","nonce":1467210441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6815","nonce":1467210441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6813","nonce":1467210441}]},"public_addr":"172.21.15.133:6809/1467210441","cluster_addr":"172.21.15.133:6811/1467210441","heartbeat_back_addr":"172.21.15.133:6815/1467210441","heartbeat_front_addr":"172.21.15.133:6813/1467210441","state":["exists","up"]},{"osd":6,"uuid":"9cf1482c-6b66-4260-a892-da6969cc8574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6819","nonce":4262819120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6821","nonce":4262819120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6825","nonce":4262819120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6823","nonce":4262819120}]},"public_addr":"172.21.15.39:6819/4262819120","cluster_addr":"172.21.15.39:6821/4262819120","heartbeat_back_addr":"172.21.15.39:6825/4262819120","heartbeat_front_addr":"172.21.15.39:6823/4262819120","state":["exists","up"]},{"osd":7,"uuid":"7b535b3c-1d36-44c9-8c22-132e6d31b0ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6817","nonce":3667650401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6819","nonce":3667650401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6823","nonce":3667650401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6821","nonce":3667650401}]},"public_addr":"172.21.15.101:6817/3667650401","cluster_addr":"172.21.15.101:6819/3667650401","heartbeat_back_addr":"172.21.15.101:6823/3667650401","heartbeat_front_addr":"172.21.15.101:6821/3667650401","state":["exists","up"]},{"osd":8,"uuid":"60376342-edba-4eb8-9907-950963e6c3e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6817","nonce":1333554117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6819","nonce":1333554117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6823","nonce":1333554117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6821","nonce":1333554117}]},"public_addr":"172.21.15.133:6817/1333554117","cluster_addr":"172.21.15.133:6819/1333554117","heartbeat_back_addr":"172.21.15.133:6823/1333554117","heartbeat_front_addr":"172.21.15.133:6821/1333554117","state":["exists","up"]},{"osd":9,"uuid":"c09c5c50-6e36-4e8e-b41c-50fb93a2ea92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6827","nonce":881304983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6829","nonce":881304983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6833","nonce":881304983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6831","nonce":881304983}]},"public_addr":"172.21.15.39:6827/881304983","cluster_addr":"172.21.15.39:6829/881304983","heartbeat_back_addr":"172.21.15.39:6833/881304983","heartbeat_front_addr":"172.21.15.39:6831/881304983","state":["exists","up"]},{"osd":10,"uuid":"4048ac5c-ffb1-4b04-85ec-6afc2add5b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6825","nonce":1823054778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6827","nonce":1823054778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6831","nonce":1823054778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6829","nonce":1823054778}]},"public_addr":"172.21.15.101:6825/1823054778","cluster_addr":"172.21.15.101:6827/1823054778","heartbeat_back_addr":"172.21.15.101:6831/1823054778","heartbeat_front_addr":"172.21.15.101:6829/1823054778","state":["exists","up"]},{"osd":11,"uuid":"334df2ca-0d0e-4450-bd11-cd8651c44e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6825","nonce":93348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6827","nonce":93348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6831","nonce":93348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6829","nonce":93348943}]},"public_addr":"172.21.15.133:6825/93348943","cluster_addr":"172.21.15.133:6827/93348943","heartbeat_back_addr":"172.21.15.133:6831/93348943","heartbeat_front_addr":"172.21.15.133:6829/93348943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:27.345253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:52.437673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:17.253223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:44.552567+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:09.454586+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:34.594288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:01.497366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:26.643829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:51.455308+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:18.528459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:43.697768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:10:09.419715+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/709665139":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/453675837":"2023-06-23T18:03:53.598476+0000","172.21.15.39:6801/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6800/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/2777224344":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/3733013426":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/466669289":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6801/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/1167071234":"2023-06-23T18:03:53.598476+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T18:10:55.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.038+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff22809b680 msgr2=0x7ff22809db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.038+0000 7ff23ffff700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff22809b680 0x7ff22809db30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff234009fd0 tx=0x7ff23400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:55.042 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.038+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 msgr2=0x7ff24010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.038+0000 7ff23ffff700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 0x7ff24010fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff230004f70 tx=0x7ff230009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:55.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 shutdown_connections 2023-06-22T18:10:55.043 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7ff22809b680 0x7ff22809db30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2401008e0 0x7ff240116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2400fed40 0x7ff24010fa00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 --2- 172.21.15.39:0/3370379771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2400fff50 0x7ff24010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.044 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 >> 172.21.15.39:0/3370379771 conn(0x7ff2400fa580 msgr2=0x7ff2401033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:55.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 shutdown_connections 2023-06-22T18:10:55.045 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.039+0000 7ff23ffff700 1 -- 172.21.15.39:0/3370379771 wait complete. 2023-06-22T18:10:55.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-22T18:06:20.190713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-06-22T18:10:55.061 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-22T18:10:55.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 -- 172.21.15.39:0/607967179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 msgr2=0x7f28e4104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/607967179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 0x7f28e4104f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0049c0 tx=0x7f28dc009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:55.277 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 -- 172.21.15.39:0/607967179 shutdown_connections 2023-06-22T18:10:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/607967179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 0x7f28e4104f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/607967179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e40fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/607967179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28e40fdcf0 0x7f28e40fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 -- 172.21.15.39:0/607967179 >> 172.21.15.39:0/607967179 conn(0x7f28e40f8200 msgr2=0x7f28e40fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 -- 172.21.15.39:0/607967179 shutdown_connections 2023-06-22T18:10:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.274+0000 7f28eaf9d700 1 -- 172.21.15.39:0/607967179 wait complete. 2023-06-22T18:10:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 Processor -- start 2023-06-22T18:10:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 -- start start 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e410f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28e40fdcf0 0x7f28e410fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 0x7f28e4116c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e4117290 con 0x7f28e40fdcf0 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e4117400 con 0x7f28e40fcae0 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.275+0000 7f28eaf9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e41175c0 con 0x7f28e40fe680 2023-06-22T18:10:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e410f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e979a700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28e40fdcf0 0x7f28e410fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e410f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:49640/0 (socket says 172.21.15.39:49640) 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 -- 172.21.15.39:0/894845952 learned_addr learned my addr 172.21.15.39:0/894845952 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28ea79c700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 0x7f28e4116c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 -- 172.21.15.39:0/894845952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 msgr2=0x7f28e4116c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 0x7f28e4116c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 -- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28e40fdcf0 msgr2=0x7f28e410fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28e40fdcf0 0x7f28e410fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 -- 172.21.15.39:0/894845952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc005040 con 0x7f28e40fcae0 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28e9f9b700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e410f6b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28e000ed30 tx=0x7f28e000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0007920 con 0x7f28e40fcae0 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28e000f040 con 0x7f28e40fcae0 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e41178a0 con 0x7f28e40fcae0 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.276+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0009380 con 0x7f28e40fcae0 2023-06-22T18:10:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.277+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e4117e10 con 0x7f28e40fcae0 2023-06-22T18:10:55.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.277+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e4108330 con 0x7f28e40fcae0 2023-06-22T18:10:55.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.279+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f28e000c910 con 0x7f28e40fcae0 2023-06-22T18:10:55.284 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.281+0000 7f28da7fc700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f28d009b680 0x7f28d009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.281+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7f28e00c48b0 con 0x7f28e40fcae0 2023-06-22T18:10:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.281+0000 7f28e979a700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f28d009b680 0x7f28d009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.281+0000 7f28e979a700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f28d009b680 0x7f28d009db30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009b80 tx=0x7f28d400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.292+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f28e004da00 con 0x7f28e40fcae0 2023-06-22T18:10:55.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.495+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28e4118030 con 0x7f28e40fcae0 2023-06-22T18:10:55.498 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.496+0000 7f28da7fc700 1 -- 172.21.15.39:0/894845952 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28e00905b0 con 0x7f28e40fcae0 2023-06-22T18:10:55.499 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2023-06-22T18:10:55.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f28d009b680 msgr2=0x7f28d009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f28d009b680 0x7f28d009db30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009b80 tx=0x7f28d400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:55.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 msgr2=0x7f28e410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.501 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e410f6b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28e000ed30 tx=0x7f28e000b040 comp rx=0 tx=0).stop 2023-06-22T18:10:55.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 shutdown_connections 2023-06-22T18:10:55.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f28d009b680 0x7f28d009db30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e40fe680 0x7f28e4116c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28e40fcae0 0x7f28e410f6b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.502 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 --2- 172.21.15.39:0/894845952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28e40fdcf0 0x7f28e410fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 >> 172.21.15.39:0/894845952 conn(0x7f28e40f8200 msgr2=0x7f28e40fa090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:55.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 shutdown_connections 2023-06-22T18:10:55.503 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.498+0000 7f28eaf9d700 1 -- 172.21.15.39:0/894845952 wait complete. 2023-06-22T18:10:55.520 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-22T18:10:55.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:55 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3379567419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:55.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:55 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/3370379771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:55.719 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 -- 172.21.15.39:0/4253736508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340fed30 msgr2=0x7fb4340ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 --2- 172.21.15.39:0/4253736508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340fed30 0x7fb4340ff140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb430004900 tx=0x7fb430009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:55.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 -- 172.21.15.39:0/4253736508 shutdown_connections 2023-06-22T18:10:55.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 --2- 172.21.15.39:0/4253736508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb434100940 0x7fb434107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 --2- 172.21.15.39:0/4253736508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340fed30 0x7fb4340ff140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.720 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 --2- 172.21.15.39:0/4253736508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4340fffb0 0x7fb434100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 -- 172.21.15.39:0/4253736508 >> 172.21.15.39:0/4253736508 conn(0x7fb4340fa3c0 msgr2=0x7fb4340fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:55.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 -- 172.21.15.39:0/4253736508 shutdown_connections 2023-06-22T18:10:55.721 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.717+0000 7fb43b19d700 1 -- 172.21.15.39:0/4253736508 wait complete. 2023-06-22T18:10:55.722 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 Processor -- start 2023-06-22T18:10:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 -- start start 2023-06-22T18:10:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4340fed30 0x7fb43410f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 0x7fb43410fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb434100940 0x7fb434116f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434117540 con 0x7fb434100940 2023-06-22T18:10:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341176b0 con 0x7fb4340fed30 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.718+0000 7fb43b19d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434117870 con 0x7fb4340fffb0 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43a99c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb434100940 0x7fb434116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 0x7fb43410fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43a19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4340fed30 0x7fb43410f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 0x7fb43410fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41258/0 (socket says 172.21.15.39:41258) 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 -- 172.21.15.39:0/1402823967 learned_addr learned my addr 172.21.15.39:0/1402823967 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 -- 172.21.15.39:0/1402823967 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4340fed30 msgr2=0x7fb43410f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4340fed30 0x7fb43410f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 -- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb434100940 msgr2=0x7fb434116f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb434100940 0x7fb434116f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 -- 172.21.15.39:0/1402823967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb424005040 con 0x7fb4340fffb0 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.719+0000 7fb43999a700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 0x7fb43410fea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb424009730 tx=0x7fb424009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.720+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424007be0 con 0x7fb4340fffb0 2023-06-22T18:10:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.720+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42400f040 con 0x7fb4340fffb0 2023-06-22T18:10:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.720+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42c0048f0 con 0x7fb4340fffb0 2023-06-22T18:10:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.720+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424013bc0 con 0x7fb4340fffb0 2023-06-22T18:10:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.720+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430005040 con 0x7fb4340fffb0 2023-06-22T18:10:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.721+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43410a5e0 con 0x7fb4340fffb0 2023-06-22T18:10:55.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.723+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fb424042bb0 con 0x7fb4340fffb0 2023-06-22T18:10:55.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.725+0000 7fb42affd700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb42009b680 0x7fb42009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:55.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.725+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fb4240c4b50 con 0x7fb4340fffb0 2023-06-22T18:10:55.727 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.725+0000 7fb43a19b700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb42009b680 0x7fb42009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.725+0000 7fb43a19b700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb42009b680 0x7fb42009db30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4300047d0 tx=0x7fb430009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.726+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fb4240907b0 con 0x7fb4340fffb0 2023-06-22T18:10:55.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:55 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3379567419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:55.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:55 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/3370379771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:55.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:55 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3379567419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-22T18:10:55.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:55 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/3370379771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:10:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.944+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb434117f60 con 0x7fb4340fffb0 2023-06-22T18:10:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.945+0000 7fb42affd700 1 -- 172.21.15.39:0/1402823967 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb4240c2e60 con 0x7fb4340fffb0 2023-06-22T18:10:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:55.948 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2023-06-22T18:10:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2023-06-22T18:10:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2023-06-22T18:10:55.948 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2023-06-22T18:10:55.949 INFO:teuthology.orchestra.run.smithi039.stdout: } 2023-06-22T18:10:55.949 INFO:teuthology.orchestra.run.smithi039.stdout:] 2023-06-22T18:10:55.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb42009b680 msgr2=0x7fb42009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb42009b680 0x7fb42009db30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4300047d0 tx=0x7fb430009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:55.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 msgr2=0x7fb43410fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:55.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 0x7fb43410fea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb424009730 tx=0x7fb424009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:55.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 shutdown_connections 2023-06-22T18:10:55.950 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fb42009b680 0x7fb42009db30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb434100940 0x7fb434116f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4340fed30 0x7fb43410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 --2- 172.21.15.39:0/1402823967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4340fffb0 0x7fb43410fea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:55.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 >> 172.21.15.39:0/1402823967 conn(0x7fb4340fa3c0 msgr2=0x7fb4340fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:55.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 shutdown_connections 2023-06-22T18:10:55.951 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:55.947+0000 7fb43b19d700 1 -- 172.21.15.39:0/1402823967 wait complete. 2023-06-22T18:10:55.968 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-22T18:10:55.969 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-22T18:10:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.178+0000 7fa6e69ff700 1 -- 172.21.15.39:0/3005703922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 msgr2=0x7fa6e00fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.178+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/3005703922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e00fd120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0004900 tx=0x7fa6d0009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 -- 172.21.15.39:0/3005703922 shutdown_connections 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/3005703922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6e00fe970 0x7fa6e0105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/3005703922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e00fd120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/3005703922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6e00fdfc0 0x7fa6e00fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 -- 172.21.15.39:0/3005703922 >> 172.21.15.39:0/3005703922 conn(0x7fa6e00f8490 msgr2=0x7fa6e00fa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:56.182 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 -- 172.21.15.39:0/3005703922 shutdown_connections 2023-06-22T18:10:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.179+0000 7fa6e69ff700 1 -- 172.21.15.39:0/3005703922 wait complete. 2023-06-22T18:10:56.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.180+0000 7fa6e69ff700 1 Processor -- start 2023-06-22T18:10:56.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.180+0000 7fa6e69ff700 1 -- start start 2023-06-22T18:10:56.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.180+0000 7fa6e69ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e010d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:56.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.180+0000 7fa6e69ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6e00fdfc0 0x7fa6e010dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:56.184 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6e69ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6e00fe970 0x7fa6e0114e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6e69ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0115440 con 0x7fa6e00fdfc0 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6e69ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01155b0 con 0x7fa6e00fcd10 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6e69ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0115770 con 0x7fa6e00fe970 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6e00fdfc0 0x7fa6e010dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e010d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e010d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:51084/0 (socket says 172.21.15.39:51084) 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 -- 172.21.15.39:0/2397553319 learned_addr learned my addr 172.21.15.39:0/2397553319 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6e4f9c700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6e00fe970 0x7fa6e0114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 -- 172.21.15.39:0/2397553319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6e00fe970 msgr2=0x7fa6e0114e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6e00fe970 0x7fa6e0114e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 -- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6e00fdfc0 msgr2=0x7fa6e010dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6e00fdfc0 0x7fa6e010dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 -- 172.21.15.39:0/2397553319 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0005040 con 0x7fa6e00fcd10 2023-06-22T18:10:56.186 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.181+0000 7fa6dffff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e010d860 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009040 tx=0x7fa6d0009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.182+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d000e040 con 0x7fa6e00fcd10 2023-06-22T18:10:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.182+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6d001c430 con 0x7fa6e00fcd10 2023-06-22T18:10:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.182+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e01159f0 con 0x7fa6e00fcd10 2023-06-22T18:10:56.187 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.182+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0012650 con 0x7fa6e00fcd10 2023-06-22T18:10:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.182+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e0115ed0 con 0x7fa6e00fcd10 2023-06-22T18:10:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.183+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e0104150 con 0x7fa6e00fcd10 2023-06-22T18:10:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.185+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fa6d0012a80 con 0x7fa6e00fcd10 2023-06-22T18:10:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.187+0000 7fa6dcff9700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa6c809b7b0 0x7fa6c809dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.187+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7123+0+0 (secure 0 0 0) 0x7fa6d00c5ce0 con 0x7fa6e00fcd10 2023-06-22T18:10:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.187+0000 7fa6df7fe700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa6c809b7b0 0x7fa6c809dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.188+0000 7fa6df7fe700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa6c809b7b0 0x7fa6c809dc60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009b80 tx=0x7fa6d400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:56.193 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.191+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fa6d0091a70 con 0x7fa6e00fcd10 2023-06-22T18:10:56.409 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:56.406+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa6e0108570 con 0x7fa6e00fcd10 2023-06-22T18:10:57.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/894845952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:57.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:57.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1402823967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-22T18:10:57.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2397553319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-22T18:10:57.195 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:56 smithi039 ceph-mon[109976]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-22T18:10:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/894845952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1402823967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-22T18:10:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.234 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2397553319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-22T18:10:57.235 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:56 smithi133 ceph-mon[116867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-22T18:10:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/894845952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-22T18:10:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1402823967' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-22T18:10:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.353 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:57.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2397553319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-22T18:10:57.354 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:56 smithi101 ceph-mon[116267]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-22T18:10:57.375 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.372+0000 7fa6dcff9700 1 -- 172.21.15.39:0/2397553319 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa6d001c5a0 con 0x7fa6e00fcd10 2023-06-22T18:10:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.377+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa6c809b7b0 msgr2=0x7fa6c809dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.377+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa6c809b7b0 0x7fa6c809dc60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009b80 tx=0x7fa6d400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.377+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 msgr2=0x7fa6e010d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.377+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e010d860 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009040 tx=0x7fa6d0009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.378+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 shutdown_connections 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.378+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fa6c809b7b0 0x7fa6c809dc60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.378+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6e00fe970 0x7fa6e0114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.378+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e00fcd10 0x7fa6e010d860 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009040 tx=0x7fa6d0009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:57.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.378+0000 7fa6e69ff700 1 --2- 172.21.15.39:0/2397553319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6e00fdfc0 0x7fa6e010dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.378+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 >> 172.21.15.39:0/2397553319 conn(0x7fa6e00f8490 msgr2=0x7fa6e01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:57.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.379+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 shutdown_connections 2023-06-22T18:10:57.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.379+0000 7fa6e69ff700 1 -- 172.21.15.39:0/2397553319 wait complete. 2023-06-22T18:10:57.384 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2023-06-22T18:10:57.400 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-22T18:10:57.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 -- 172.21.15.39:0/1828125592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fcaf0 msgr2=0x7fd4440fcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:57.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/1828125592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fcaf0 0x7fd4440fcf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4400049c0 tx=0x7fd440009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:57.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 -- 172.21.15.39:0/1828125592 shutdown_connections 2023-06-22T18:10:57.600 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/1828125592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4440fe690 0x7fd444104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/1828125592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fcaf0 0x7fd4440fcf00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/1828125592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4440fdd00 0x7fd4440fe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 -- 172.21.15.39:0/1828125592 >> 172.21.15.39:0/1828125592 conn(0x7fd4440f8210 msgr2=0x7fd4440fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:57.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 -- 172.21.15.39:0/1828125592 shutdown_connections 2023-06-22T18:10:57.601 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.597+0000 7fd44ad9d700 1 -- 172.21.15.39:0/1828125592 wait complete. 2023-06-22T18:10:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 Processor -- start 2023-06-22T18:10:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 -- start start 2023-06-22T18:10:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4440fcaf0 0x7fd444071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 0x7fd44406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4440fe690 0x7fd4440702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd444070a30 con 0x7fd4440fe690 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd444070ba0 con 0x7fd4440fdd00 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44ad9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4441178d0 con 0x7fd4440fcaf0 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 0x7fd44406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44959a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 0x7fd44406fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:51104/0 (socket says 172.21.15.39:51104) 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd449d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4440fcaf0 0x7fd444071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:57.604 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44959a700 1 -- 172.21.15.39:0/2958526699 learned_addr learned my addr 172.21.15.39:0/2958526699 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.598+0000 7fd44a59c700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4440fe690 0x7fd4440702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44959a700 1 -- 172.21.15.39:0/2958526699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4440fcaf0 msgr2=0x7fd444071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44959a700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4440fcaf0 0x7fd444071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44959a700 1 -- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4440fe690 msgr2=0x7fd4440702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44959a700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4440fe690 0x7fd4440702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44959a700 1 -- 172.21.15.39:0/2958526699 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440005040 con 0x7fd4440fdd00 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44959a700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 0x7fd44406fd80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd43400ebc0 tx=0x7fd43400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434007cc0 con 0x7fd4440fdd00 2023-06-22T18:10:57.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd43400f040 con 0x7fd4440fdd00 2023-06-22T18:10:57.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd444117b20 con 0x7fd4440fdd00 2023-06-22T18:10:57.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4340094a0 con 0x7fd4440fdd00 2023-06-22T18:10:57.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.599+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd444118010 con 0x7fd4440fdd00 2023-06-22T18:10:57.606 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.600+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd444103a00 con 0x7fd4440fdd00 2023-06-22T18:10:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.603+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fd43400c600 con 0x7fd4440fdd00 2023-06-22T18:10:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.604+0000 7fd43a7fc700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd43009b680 0x7fd43009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.604+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7493+0+0 (secure 0 0 0) 0x7fd4340c4790 con 0x7fd4440fdd00 2023-06-22T18:10:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.604+0000 7fd449d9b700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd43009b680 0x7fd43009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.605+0000 7fd449d9b700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd43009b680 0x7fd43009db30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd440004890 tx=0x7fd440009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.606+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fd43404da00 con 0x7fd4440fdd00 2023-06-22T18:10:57.817 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:57.814+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd444108330 con 0x7fd4440fdd00 2023-06-22T18:10:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.383+0000 7fd43a7fc700 1 -- 172.21.15.39:0/2958526699 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd4340190d0 con 0x7fd4440fdd00 2023-06-22T18:10:58.392 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.390+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd43009b680 msgr2=0x7fd43009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.390+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd43009b680 0x7fd43009db30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd440004890 tx=0x7fd440009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.390+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 msgr2=0x7fd44406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.390+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 0x7fd44406fd80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd43400ebc0 tx=0x7fd43400b040 comp rx=0 tx=0).stop 2023-06-22T18:10:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.391+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 shutdown_connections 2023-06-22T18:10:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.391+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fd43009b680 0x7fd43009db30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.391+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4440fe690 0x7fd4440702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.391+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4440fcaf0 0x7fd444071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.391+0000 7fd44ad9d700 1 --2- 172.21.15.39:0/2958526699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4440fdd00 0x7fd44406fd80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.391+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 >> 172.21.15.39:0/2958526699 conn(0x7fd4440f8210 msgr2=0x7fd4440fa490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.392+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 shutdown_connections 2023-06-22T18:10:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.395+0000 7fd44ad9d700 1 -- 172.21.15.39:0/2958526699 wait complete. 2023-06-22T18:10:58.405 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2023-06-22T18:10:58.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-22T18:10:58.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:58.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:58.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-22T18:10:58.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: osdmap e82: 12 total, 12 up, 12 in 2023-06-22T18:10:58.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:58.605 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:58 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2958526699' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-22T18:10:58.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 -- 172.21.15.39:0/1445462992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8102740 msgr2=0x7f1ed8104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/1445462992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8102740 0x7f1ed8104bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0004ca0 tx=0x7f1ed0009ca0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 -- 172.21.15.39:0/1445462992 shutdown_connections 2023-06-22T18:10:58.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/1445462992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8102740 0x7f1ed8104bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.639 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/1445462992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed8065c10 0x7f1ed80ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/1445462992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ffc50 0x7f1ed8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 -- 172.21.15.39:0/1445462992 >> 172.21.15.39:0/1445462992 conn(0x7f1ed80f7100 msgr2=0x7f1ed80f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 -- 172.21.15.39:0/1445462992 shutdown_connections 2023-06-22T18:10:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.636+0000 7f1edc9fe700 1 -- 172.21.15.39:0/1445462992 wait complete. 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 Processor -- start 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 -- start start 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 0x7f1ed80fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed80ffc50 0x7f1ed80fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed8102740 0x7f1ed80fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80fc2c0 con 0x7f1ed80ffc50 2023-06-22T18:10:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80fc430 con 0x7f1ed8065c10 2023-06-22T18:10:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.637+0000 7f1edc9fe700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8117700 con 0x7f1ed8102740 2023-06-22T18:10:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 0x7f1ed80fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 0x7f1ed80fcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:51120/0 (socket says 172.21.15.39:51120) 2023-06-22T18:10:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 -- 172.21.15.39:0/362771951 learned_addr learned my addr 172.21.15.39:0/362771951 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed659c700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed80ffc50 0x7f1ed80fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed759e700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed8102740 0x7f1ed80fbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 -- 172.21.15.39:0/362771951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed8102740 msgr2=0x7f1ed80fbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed8102740 0x7f1ed80fbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 -- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed80ffc50 msgr2=0x7f1ed80fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed80ffc50 0x7f1ed80fb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 -- 172.21.15.39:0/362771951 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed00048f0 con 0x7f1ed8065c10 2023-06-22T18:10:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed659c700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed80ffc50 0x7f1ed80fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-22T18:10:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ed6d9d700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 0x7f1ed80fcfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800ebf0 tx=0x7f1ec800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8007cc0 con 0x7f1ed8065c10 2023-06-22T18:10:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8117950 con 0x7f1ed8065c10 2023-06-22T18:10:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ec800f040 con 0x7f1ed8065c10 2023-06-22T18:10:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.638+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800c8f0 con 0x7f1ed8065c10 2023-06-22T18:10:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.639+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed8117ec0 con 0x7f1ed8065c10 2023-06-22T18:10:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.640+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed80f8c30 con 0x7f1ed8065c10 2023-06-22T18:10:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.641+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f1ec800cd20 con 0x7f1ed8065c10 2023-06-22T18:10:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.643+0000 7f1ebf7fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1ec009b680 0x7f1ec009db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.643+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7875+0+0 (secure 0 0 0) 0x7f1ec80c48f0 con 0x7f1ed8065c10 2023-06-22T18:10:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.643+0000 7f1ed659c700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1ec009b680 0x7f1ec009db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.643+0000 7f1ed659c700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1ec009b680 0x7f1ec009db30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007fd0 tx=0x7f1ecc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.648+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f1ec804da00 con 0x7f1ed8065c10 2023-06-22T18:10:58.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-22T18:10:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: osdmap e82: 12 total, 12 up, 12 in 2023-06-22T18:10:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:58.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:58 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2958526699' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-22T18:10:58.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-22T18:10:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-22T18:10:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:10:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-22T18:10:58.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: osdmap e82: 12 total, 12 up, 12 in 2023-06-22T18:10:58.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-22T18:10:58.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:58 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2958526699' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-22T18:10:58.915 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.913+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1ed8108330 con 0x7f1ed8065c10 2023-06-22T18:10:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.928+0000 7f1ebf7fe700 1 -- 172.21.15.39:0/362771951 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1ec8019070 con 0x7f1ed8065c10 2023-06-22T18:10:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.934+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1ec009b680 msgr2=0x7f1ec009db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.934+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1ec009b680 0x7f1ec009db30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007fd0 tx=0x7f1ecc00b040 comp rx=0 tx=0).stop 2023-06-22T18:10:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.934+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 msgr2=0x7f1ed80fcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.934+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 0x7f1ed80fcfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800ebf0 tx=0x7f1ec800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 shutdown_connections 2023-06-22T18:10:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f1ec009b680 0x7f1ec009db30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed8102740 0x7f1ed80fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ed8065c10 0x7f1ed80fcfb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 --2- 172.21.15.39:0/362771951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed80ffc50 0x7f1ed80fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 >> 172.21.15.39:0/362771951 conn(0x7f1ed80f7100 msgr2=0x7f1ed80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 shutdown_connections 2023-06-22T18:10:58.939 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:58.935+0000 7f1edc9fe700 1 -- 172.21.15.39:0/362771951 wait complete. 2023-06-22T18:10:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2023-06-22T18:10:58.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T18:10:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 -- 172.21.15.39:0/3872206312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 msgr2=0x7f074c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 --2- 172.21.15.39:0/3872206312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 0x7f074c1051d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0744004ca0 tx=0x7f0744009ca0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 -- 172.21.15.39:0/3872206312 shutdown_connections 2023-06-22T18:10:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 --2- 172.21.15.39:0/3872206312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 0x7f074c1051d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 --2- 172.21.15.39:0/3872206312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f074c0fccd0 0x7f074c0fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 --2- 172.21.15.39:0/3872206312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fdf80 0x7f074c0fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 -- 172.21.15.39:0/3872206312 >> 172.21.15.39:0/3872206312 conn(0x7f074c0f83f0 msgr2=0x7f074c0fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.164+0000 7f07531ff700 1 -- 172.21.15.39:0/3872206312 shutdown_connections 2023-06-22T18:10:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.165+0000 7f07531ff700 1 -- 172.21.15.39:0/3872206312 wait complete. 2023-06-22T18:10:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.165+0000 7f07531ff700 1 Processor -- start 2023-06-22T18:10:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.165+0000 7f07531ff700 1 -- start start 2023-06-22T18:10:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f07531ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 0x7f074c10d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f07531ff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f074c0fdf80 0x7f074c10dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f07531ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 0x7f074c114d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f07531ff700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c115320 con 0x7f074c0fdf80 2023-06-22T18:10:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f07531ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c115490 con 0x7f074c0fccd0 2023-06-22T18:10:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f07531ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c115650 con 0x7f074c0fe930 2023-06-22T18:10:59.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 0x7f074c10d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f074bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f074c0fdf80 0x7f074c10dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.176 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 0x7f074c10d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:51148/0 (socket says 172.21.15.39:51148) 2023-06-22T18:10:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 -- 172.21.15.39:0/1541002894 learned_addr learned my addr 172.21.15.39:0/1541002894 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f075179c700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 0x7f074c114d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 -- 172.21.15.39:0/1541002894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 msgr2=0x7f074c114d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 0x7f074c114d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 -- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f074c0fdf80 msgr2=0x7f074c10dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f074c0fdf80 0x7f074c10dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.166+0000 7f0750f9b700 1 -- 172.21.15.39:0/1541002894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0740005040 con 0x7f074c0fccd0 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.167+0000 7f0750f9b700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 0x7f074c10d740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0740009730 tx=0x7f0740009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.167+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740007be0 con 0x7f074c0fccd0 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.167+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07440048f0 con 0x7f074c0fccd0 2023-06-22T18:10:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.167+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074000f040 con 0x7f074c0fccd0 2023-06-22T18:10:59.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.167+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740013bc0 con 0x7f074c0fccd0 2023-06-22T18:10:59.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.167+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c115cb0 con 0x7f074c0fccd0 2023-06-22T18:10:59.179 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.168+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c104090 con 0x7f074c0fccd0 2023-06-22T18:10:59.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.170+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f074000e840 con 0x7f074c0fccd0 2023-06-22T18:10:59.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.171+0000 7f07497fa700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f073c09b680 0x7f073c09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.171+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f07400c4b20 con 0x7f074c0fccd0 2023-06-22T18:10:59.180 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.171+0000 7f074bfff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f073c09b680 0x7f073c09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.172+0000 7f074bfff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f073c09b680 0x7f073c09db30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0738009b80 tx=0x7f073800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:59.181 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.175+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f0740090430 con 0x7f074c0fccd0 2023-06-22T18:10:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.386+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f074c108530 con 0x7f074c0fccd0 2023-06-22T18:10:59.390 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.387+0000 7f07497fa700 1 -- 172.21.15.39:0/1541002894 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21358 (secure 0 0 0) 0x7f0740093a50 con 0x7f074c0fccd0 2023-06-22T18:10:59.391 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:59.391 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","created":"2023-06-22T18:02:53.160326+0000","modified":"2023-06-22T18:10:58.914595+0000","last_up_change":"2023-06-22T18:10:10.873966+0000","last_in_change":"2023-06-22T18:09:51.312474+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T18:06:20.190713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-22T18:10:56.927405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-22T18:10:58.332891+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f55d12a1-86b5-42a6-ad35-d5eaf3d99c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6803","nonce":471291770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6805","nonce":471291770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6809","nonce":471291770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6807","nonce":471291770}]},"public_addr":"172.21.15.39:6803/471291770","cluster_addr":"172.21.15.39:6805/471291770","heartbeat_back_addr":"172.21.15.39:6809/471291770","heartbeat_front_addr":"172.21.15.39:6807/471291770","state":["exists","up"]},{"osd":1,"uuid":"4d8fb427-c4b2-45c3-ac42-dea08c42e4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6801","nonce":840111728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6803","nonce":840111728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6807","nonce":840111728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6805","nonce":840111728}]},"public_addr":"172.21.15.101:6801/840111728","cluster_addr":"172.21.15.101:6803/840111728","heartbeat_back_addr":"172.21.15.101:6807/840111728","heartbeat_front_addr":"172.21.15.101:6805/840111728","state":["exists","up"]},{"osd":2,"uuid":"9c714cec-14fc-4aed-b3f1-f0773edd197c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6801","nonce":3165338416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6803","nonce":3165338416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6807","nonce":3165338416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6805","nonce":3165338416}]},"public_addr":"172.21.15.133:6801/3165338416","cluster_addr":"172.21.15.133:6803/3165338416","heartbeat_back_addr":"172.21.15.133:6807/3165338416","heartbeat_front_addr":"172.21.15.133:6805/3165338416","state":["exists","up"]},{"osd":3,"uuid":"71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6811","nonce":3695104438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6813","nonce":3695104438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6817","nonce":3695104438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6815","nonce":3695104438}]},"public_addr":"172.21.15.39:6811/3695104438","cluster_addr":"172.21.15.39:6813/3695104438","heartbeat_back_addr":"172.21.15.39:6817/3695104438","heartbeat_front_addr":"172.21.15.39:6815/3695104438","state":["exists","up"]},{"osd":4,"uuid":"c9ecace5-1032-4f25-a5f3-84ec819c83ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6809","nonce":2726745315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6811","nonce":2726745315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6815","nonce":2726745315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6813","nonce":2726745315}]},"public_addr":"172.21.15.101:6809/2726745315","cluster_addr":"172.21.15.101:6811/2726745315","heartbeat_back_addr":"172.21.15.101:6815/2726745315","heartbeat_front_addr":"172.21.15.101:6813/2726745315","state":["exists","up"]},{"osd":5,"uuid":"379ed499-68d9-4032-bafd-31cb02995388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6809","nonce":1467210441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6811","nonce":1467210441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6815","nonce":1467210441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6813","nonce":1467210441}]},"public_addr":"172.21.15.133:6809/1467210441","cluster_addr":"172.21.15.133:6811/1467210441","heartbeat_back_addr":"172.21.15.133:6815/1467210441","heartbeat_front_addr":"172.21.15.133:6813/1467210441","state":["exists","up"]},{"osd":6,"uuid":"9cf1482c-6b66-4260-a892-da6969cc8574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6819","nonce":4262819120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6821","nonce":4262819120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6825","nonce":4262819120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6823","nonce":4262819120}]},"public_addr":"172.21.15.39:6819/4262819120","cluster_addr":"172.21.15.39:6821/4262819120","heartbeat_back_addr":"172.21.15.39:6825/4262819120","heartbeat_front_addr":"172.21.15.39:6823/4262819120","state":["exists","up"]},{"osd":7,"uuid":"7b535b3c-1d36-44c9-8c22-132e6d31b0ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6817","nonce":3667650401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6819","nonce":3667650401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6823","nonce":3667650401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6821","nonce":3667650401}]},"public_addr":"172.21.15.101:6817/3667650401","cluster_addr":"172.21.15.101:6819/3667650401","heartbeat_back_addr":"172.21.15.101:6823/3667650401","heartbeat_front_addr":"172.21.15.101:6821/3667650401","state":["exists","up"]},{"osd":8,"uuid":"60376342-edba-4eb8-9907-950963e6c3e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6817","nonce":1333554117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6819","nonce":1333554117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6823","nonce":1333554117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6821","nonce":1333554117}]},"public_addr":"172.21.15.133:6817/1333554117","cluster_addr":"172.21.15.133:6819/1333554117","heartbeat_back_addr":"172.21.15.133:6823/1333554117","heartbeat_front_addr":"172.21.15.133:6821/1333554117","state":["exists","up"]},{"osd":9,"uuid":"c09c5c50-6e36-4e8e-b41c-50fb93a2ea92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6827","nonce":881304983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6829","nonce":881304983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6833","nonce":881304983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6831","nonce":881304983}]},"public_addr":"172.21.15.39:6827/881304983","cluster_addr":"172.21.15.39:6829/881304983","heartbeat_back_addr":"172.21.15.39:6833/881304983","heartbeat_front_addr":"172.21.15.39:6831/881304983","state":["exists","up"]},{"osd":10,"uuid":"4048ac5c-ffb1-4b04-85ec-6afc2add5b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6825","nonce":1823054778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6827","nonce":1823054778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6831","nonce":1823054778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6829","nonce":1823054778}]},"public_addr":"172.21.15.101:6825/1823054778","cluster_addr":"172.21.15.101:6827/1823054778","heartbeat_back_addr":"172.21.15.101:6831/1823054778","heartbeat_front_addr":"172.21.15.101:6829/1823054778","state":["exists","up"]},{"osd":11,"uuid":"334df2ca-0d0e-4450-bd11-cd8651c44e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6825","nonce":93348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6827","nonce":93348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6831","nonce":93348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6829","nonce":93348943}]},"public_addr":"172.21.15.133:6825/93348943","cluster_addr":"172.21.15.133:6827/93348943","heartbeat_back_addr":"172.21.15.133:6831/93348943","heartbeat_front_addr":"172.21.15.133:6829/93348943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:27.345253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:52.437673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:17.253223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:44.552567+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:09.454586+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:34.594288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:01.497366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:26.643829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:51.455308+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:18.528459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:43.697768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:10:09.419715+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/709665139":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/453675837":"2023-06-23T18:03:53.598476+0000","172.21.15.39:6801/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6800/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/2777224344":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/3733013426":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/466669289":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6801/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/1167071234":"2023-06-23T18:03:53.598476+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T18:10:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f073c09b680 msgr2=0x7f073c09db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f073c09b680 0x7f073c09db30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0738009b80 tx=0x7f073800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 msgr2=0x7f074c10d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 0x7f074c10d740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0740009730 tx=0x7f0740009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 shutdown_connections 2023-06-22T18:10:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f073c09b680 0x7f073c09db30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f074c0fe930 0x7f074c114d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f074c0fccd0 0x7f074c10d740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 --2- 172.21.15.39:0/1541002894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f074c0fdf80 0x7f074c10dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 >> 172.21.15.39:0/1541002894 conn(0x7f074c0f83f0 msgr2=0x7f074c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 shutdown_connections 2023-06-22T18:10:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.390+0000 7f07531ff700 1 -- 172.21.15.39:0/1541002894 wait complete. 2023-06-22T18:10:59.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-22T18:10:59.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:59.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-22T18:10:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-22T18:10:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: osdmap e83: 12 total, 12 up, 12 in 2023-06-22T18:10:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/362771951' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-22T18:10:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-22T18:10:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-22T18:10:59.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: osdmap e84: 12 total, 12 up, 12 in 2023-06-22T18:10:59.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-22T18:10:59.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-22T18:10:59.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:10:59 smithi101 ceph-mon[116267]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-22T18:10:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 -- 172.21.15.39:0/2064682247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fcb10 msgr2=0x7f35c80fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2064682247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fcb10 0x7f35c80fcf20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35c40049c0 tx=0x7f35c4009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 -- 172.21.15.39:0/2064682247 shutdown_connections 2023-06-22T18:10:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2064682247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35c80fe6b0 0x7f35c8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2064682247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fcb10 0x7f35c80fcf20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2064682247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80fdd20 0x7f35c80fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 -- 172.21.15.39:0/2064682247 >> 172.21.15.39:0/2064682247 conn(0x7f35c80f8270 msgr2=0x7f35c80fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:59.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 -- 172.21.15.39:0/2064682247 shutdown_connections 2023-06-22T18:10:59.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.604+0000 7f35ce39d700 1 -- 172.21.15.39:0/2064682247 wait complete. 2023-06-22T18:10:59.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 Processor -- start 2023-06-22T18:10:59.609 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 -- start start 2023-06-22T18:10:59.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35c80fcb10 0x7f35c8071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 0x7f35c806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.611 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80fe6b0 0x7f35c80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8070a30 con 0x7f35c80fcb10 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8070ba0 con 0x7f35c80fdd20 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ce39d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c8117950 con 0x7f35c80fe6b0 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ccb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 0x7f35c806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35cd39b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35c80fcb10 0x7f35c8071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ccb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 0x7f35c806fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.39:51172/0 (socket says 172.21.15.39:51172) 2023-06-22T18:10:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.605+0000 7f35ccb9a700 1 -- 172.21.15.39:0/2069292772 learned_addr learned my addr 172.21.15.39:0/2069292772 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35cdb9c700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80fe6b0 0x7f35c80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ccb9a700 1 -- 172.21.15.39:0/2069292772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80fe6b0 msgr2=0x7f35c80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ccb9a700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80fe6b0 0x7f35c80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ccb9a700 1 -- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35c80fcb10 msgr2=0x7f35c8071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ccb9a700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35c80fcb10 0x7f35c8071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ccb9a700 1 -- 172.21.15.39:0/2069292772 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c4005040 con 0x7f35c80fdd20 2023-06-22T18:10:59.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ccb9a700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 0x7f35c806fd80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35b800ebc0 tx=0x7f35b800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8007cc0 con 0x7f35c80fdd20 2023-06-22T18:10:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35b800f040 con 0x7f35c80fdd20 2023-06-22T18:10:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b80094a0 con 0x7f35c80fdd20 2023-06-22T18:10:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.606+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c8117ba0 con 0x7f35c80fdd20 2023-06-22T18:10:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.607+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c810d510 con 0x7f35c80fdd20 2023-06-22T18:10:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.608+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c8103ff0 con 0x7f35c80fdd20 2023-06-22T18:10:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.610+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f35b800c600 con 0x7f35c80fdd20 2023-06-22T18:10:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.611+0000 7f35bdffb700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f35b409b700 0x7f35b409dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:10:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.611+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f35b80c4800 con 0x7f35c80fdd20 2023-06-22T18:10:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.611+0000 7f35cd39b700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f35b409b700 0x7f35b409dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:10:59.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.612+0000 7f35cd39b700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f35b409b700 0x7f35b409dbb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35c4007f50 tx=0x7f35c4009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:10:59.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.615+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f35b804da00 con 0x7f35c80fdd20 2023-06-22T18:10:59.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:59.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-22T18:10:59.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-22T18:10:59.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: osdmap e83: 12 total, 12 up, 12 in 2023-06-22T18:10:59.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/362771951' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-22T18:10:59.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-22T18:10:59.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-22T18:10:59.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: osdmap e84: 12 total, 12 up, 12 in 2023-06-22T18:10:59.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-22T18:10:59.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-22T18:10:59.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:10:59 smithi039 ceph-mon[109976]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-22T18:10:59.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: pgmap v284: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-22T18:10:59.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-22T18:10:59.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-22T18:10:59.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: osdmap e83: 12 total, 12 up, 12 in 2023-06-22T18:10:59.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/362771951' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-22T18:10:59.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-22T18:10:59.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-22T18:10:59.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: osdmap e84: 12 total, 12 up, 12 in 2023-06-22T18:10:59.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-22T18:10:59.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-22T18:10:59.736 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:10:59 smithi133 ceph-mon[116867]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-22T18:10:59.842 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.840+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35c8108350 con 0x7f35c80fdd20 2023-06-22T18:10:59.843 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.841+0000 7f35bdffb700 1 -- 172.21.15.39:0/2069292772 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21358 (secure 0 0 0) 0x7f35b8019090 con 0x7f35c80fdd20 2023-06-22T18:10:59.844 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-06-22T18:10:59.844 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"fsid":"d572fea0-1126-11ee-9b2b-001a4aab830c","created":"2023-06-22T18:02:53.160326+0000","modified":"2023-06-22T18:10:58.914595+0000","last_up_change":"2023-06-22T18:10:10.873966+0000","last_in_change":"2023-06-22T18:09:51.312474+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-22T18:06:20.190713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-22T18:10:56.927405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-22T18:10:58.332891+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f55d12a1-86b5-42a6-ad35-d5eaf3d99c62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6803","nonce":471291770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6805","nonce":471291770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6809","nonce":471291770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":471291770},{"type":"v1","addr":"172.21.15.39:6807","nonce":471291770}]},"public_addr":"172.21.15.39:6803/471291770","cluster_addr":"172.21.15.39:6805/471291770","heartbeat_back_addr":"172.21.15.39:6809/471291770","heartbeat_front_addr":"172.21.15.39:6807/471291770","state":["exists","up"]},{"osd":1,"uuid":"4d8fb427-c4b2-45c3-ac42-dea08c42e4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6801","nonce":840111728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6803","nonce":840111728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6807","nonce":840111728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":840111728},{"type":"v1","addr":"172.21.15.101:6805","nonce":840111728}]},"public_addr":"172.21.15.101:6801/840111728","cluster_addr":"172.21.15.101:6803/840111728","heartbeat_back_addr":"172.21.15.101:6807/840111728","heartbeat_front_addr":"172.21.15.101:6805/840111728","state":["exists","up"]},{"osd":2,"uuid":"9c714cec-14fc-4aed-b3f1-f0773edd197c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6801","nonce":3165338416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6803","nonce":3165338416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6807","nonce":3165338416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3165338416},{"type":"v1","addr":"172.21.15.133:6805","nonce":3165338416}]},"public_addr":"172.21.15.133:6801/3165338416","cluster_addr":"172.21.15.133:6803/3165338416","heartbeat_back_addr":"172.21.15.133:6807/3165338416","heartbeat_front_addr":"172.21.15.133:6805/3165338416","state":["exists","up"]},{"osd":3,"uuid":"71bc1d1c-c2dd-416a-90f1-1b1cd0b9e88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6811","nonce":3695104438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6813","nonce":3695104438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6817","nonce":3695104438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":3695104438},{"type":"v1","addr":"172.21.15.39:6815","nonce":3695104438}]},"public_addr":"172.21.15.39:6811/3695104438","cluster_addr":"172.21.15.39:6813/3695104438","heartbeat_back_addr":"172.21.15.39:6817/3695104438","heartbeat_front_addr":"172.21.15.39:6815/3695104438","state":["exists","up"]},{"osd":4,"uuid":"c9ecace5-1032-4f25-a5f3-84ec819c83ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6809","nonce":2726745315}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6811","nonce":2726745315}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6815","nonce":2726745315}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":2726745315},{"type":"v1","addr":"172.21.15.101:6813","nonce":2726745315}]},"public_addr":"172.21.15.101:6809/2726745315","cluster_addr":"172.21.15.101:6811/2726745315","heartbeat_back_addr":"172.21.15.101:6815/2726745315","heartbeat_front_addr":"172.21.15.101:6813/2726745315","state":["exists","up"]},{"osd":5,"uuid":"379ed499-68d9-4032-bafd-31cb02995388","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6809","nonce":1467210441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6811","nonce":1467210441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6815","nonce":1467210441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":1467210441},{"type":"v1","addr":"172.21.15.133:6813","nonce":1467210441}]},"public_addr":"172.21.15.133:6809/1467210441","cluster_addr":"172.21.15.133:6811/1467210441","heartbeat_back_addr":"172.21.15.133:6815/1467210441","heartbeat_front_addr":"172.21.15.133:6813/1467210441","state":["exists","up"]},{"osd":6,"uuid":"9cf1482c-6b66-4260-a892-da6969cc8574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6819","nonce":4262819120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6821","nonce":4262819120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6825","nonce":4262819120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":4262819120},{"type":"v1","addr":"172.21.15.39:6823","nonce":4262819120}]},"public_addr":"172.21.15.39:6819/4262819120","cluster_addr":"172.21.15.39:6821/4262819120","heartbeat_back_addr":"172.21.15.39:6825/4262819120","heartbeat_front_addr":"172.21.15.39:6823/4262819120","state":["exists","up"]},{"osd":7,"uuid":"7b535b3c-1d36-44c9-8c22-132e6d31b0ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6817","nonce":3667650401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6819","nonce":3667650401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6823","nonce":3667650401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":3667650401},{"type":"v1","addr":"172.21.15.101:6821","nonce":3667650401}]},"public_addr":"172.21.15.101:6817/3667650401","cluster_addr":"172.21.15.101:6819/3667650401","heartbeat_back_addr":"172.21.15.101:6823/3667650401","heartbeat_front_addr":"172.21.15.101:6821/3667650401","state":["exists","up"]},{"osd":8,"uuid":"60376342-edba-4eb8-9907-950963e6c3e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6817","nonce":1333554117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6819","nonce":1333554117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6823","nonce":1333554117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1333554117},{"type":"v1","addr":"172.21.15.133:6821","nonce":1333554117}]},"public_addr":"172.21.15.133:6817/1333554117","cluster_addr":"172.21.15.133:6819/1333554117","heartbeat_back_addr":"172.21.15.133:6823/1333554117","heartbeat_front_addr":"172.21.15.133:6821/1333554117","state":["exists","up"]},{"osd":9,"uuid":"c09c5c50-6e36-4e8e-b41c-50fb93a2ea92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6827","nonce":881304983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6829","nonce":881304983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6833","nonce":881304983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":881304983},{"type":"v1","addr":"172.21.15.39:6831","nonce":881304983}]},"public_addr":"172.21.15.39:6827/881304983","cluster_addr":"172.21.15.39:6829/881304983","heartbeat_back_addr":"172.21.15.39:6833/881304983","heartbeat_front_addr":"172.21.15.39:6831/881304983","state":["exists","up"]},{"osd":10,"uuid":"4048ac5c-ffb1-4b04-85ec-6afc2add5b4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6825","nonce":1823054778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6827","nonce":1823054778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6831","nonce":1823054778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":1823054778},{"type":"v1","addr":"172.21.15.101:6829","nonce":1823054778}]},"public_addr":"172.21.15.101:6825/1823054778","cluster_addr":"172.21.15.101:6827/1823054778","heartbeat_back_addr":"172.21.15.101:6831/1823054778","heartbeat_front_addr":"172.21.15.101:6829/1823054778","state":["exists","up"]},{"osd":11,"uuid":"334df2ca-0d0e-4450-bd11-cd8651c44e3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6825","nonce":93348943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6827","nonce":93348943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6831","nonce":93348943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":93348943},{"type":"v1","addr":"172.21.15.133:6829","nonce":93348943}]},"public_addr":"172.21.15.133:6825/93348943","cluster_addr":"172.21.15.133:6827/93348943","heartbeat_back_addr":"172.21.15.133:6831/93348943","heartbeat_front_addr":"172.21.15.133:6829/93348943","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:27.345253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:05:52.437673+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:17.253223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:06:44.552567+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:09.454586+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:07:34.594288+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:01.497366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:26.643829+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:08:51.455308+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:18.528459+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:09:43.697768+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-22T18:10:09.419715+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/709665139":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/453675837":"2023-06-23T18:03:53.598476+0000","172.21.15.39:6801/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6800/2590139173":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/2777224344":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/3733013426":"2023-06-23T18:03:28.099690+0000","172.21.15.39:0/466669289":"2023-06-23T18:03:28.099690+0000","172.21.15.39:6801/349518747":"2023-06-23T18:03:53.598476+0000","172.21.15.39:0/1167071234":"2023-06-23T18:03:53.598476+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-22T18:10:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f35b409b700 msgr2=0x7f35b409dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f35b409b700 0x7f35b409dbb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35c4007f50 tx=0x7f35c4009e30 comp rx=0 tx=0).stop 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 msgr2=0x7f35c806fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 0x7f35c806fd80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35b800ebc0 tx=0x7f35b800b040 comp rx=0 tx=0).stop 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 shutdown_connections 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f35b409b700 0x7f35b409dbb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80fe6b0 0x7f35c80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35c80fcb10 0x7f35c8071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.848 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 --2- 172.21.15.39:0/2069292772 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c80fdd20 0x7f35c806fd80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:10:59.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 >> 172.21.15.39:0/2069292772 conn(0x7f35c80f8270 msgr2=0x7f35c8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:10:59.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 shutdown_connections 2023-06-22T18:10:59.849 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:10:59.843+0000 7f35ce39d700 1 -- 172.21.15.39:0/2069292772 wait complete. 2023-06-22T18:10:59.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-22T18:11:00.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 -- 172.21.15.39:0/2886436779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc0fd660 msgr2=0x7f22dc0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:11:00.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 --2- 172.21.15.39:0/2886436779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc0fd660 0x7f22dc0ffa40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22d80049c0 tx=0x7f22d8009e30 comp rx=0 tx=0).stop 2023-06-22T18:11:00.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 -- 172.21.15.39:0/2886436779 shutdown_connections 2023-06-22T18:11:00.066 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 --2- 172.21.15.39:0/2886436779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc1028a0 0x7f22dc104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 --2- 172.21.15.39:0/2886436779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc0fd660 0x7f22dc0ffa40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 --2- 172.21.15.39:0/2886436779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fff80 0x7f22dc102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 -- 172.21.15.39:0/2886436779 >> 172.21.15.39:0/2886436779 conn(0x7f22dc0f7160 msgr2=0x7f22dc0f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:11:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 -- 172.21.15.39:0/2886436779 shutdown_connections 2023-06-22T18:11:00.067 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.063+0000 7f22e319d700 1 -- 172.21.15.39:0/2886436779 wait complete. 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 Processor -- start 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 -- start start 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fd660 0x7f22dc10f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0fff80 0x7f22dc10ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc1028a0 0x7f22dc116fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1175b0 con 0x7f22dc0fd660 2023-06-22T18:11:00.069 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc117720 con 0x7f22dc1028a0 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.064+0000 7f22e319d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc1178e0 con 0x7f22dc0fff80 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e219b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fd660 0x7f22dc10f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0fff80 0x7f22dc10ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e219b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fd660 0x7f22dc10f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37954/0 (socket says 172.21.15.39:37954) 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e219b700 1 -- 172.21.15.39:0/3942427139 learned_addr learned my addr 172.21.15.39:0/3942427139 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e299c700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc1028a0 0x7f22dc116fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.070 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 -- 172.21.15.39:0/3942427139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc1028a0 msgr2=0x7f22dc116fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22dc1028a0 0x7f22dc116fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 -- 172.21.15.39:0/3942427139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fd660 msgr2=0x7f22dc10f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fd660 0x7f22dc10f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 -- 172.21.15.39:0/3942427139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8005040 con 0x7f22dc0fff80 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e219b700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22dc0fd660 0x7f22dc10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22e199a700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22dc0fff80 0x7f22dc10ff10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009fd0 tx=0x7f22cc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:11:00.071 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.065+0000 7f22d2ffd700 1 -- 172.21.15.39:0/3942427139 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc007cc0 con 0x7f22dc0fff80 2023-06-22T18:11:00.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.066+0000 7f22e319d700 1 -- 172.21.15.39:0/3942427139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc117b60 con 0x7f22dc0fff80 2023-06-22T18:11:00.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.066+0000 7f22d2ffd700 1 -- 172.21.15.39:0/3942427139 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22cc00f040 con 0x7f22dc0fff80 2023-06-22T18:11:00.072 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.066+0000 7f22d2ffd700 1 -- 172.21.15.39:0/3942427139 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00c980 con 0x7f22dc0fff80 2023-06-22T18:11:00.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.066+0000 7f22e319d700 1 -- 172.21.15.39:0/3942427139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc118050 con 0x7f22dc0fff80 2023-06-22T18:11:00.073 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.067+0000 7f22e319d700 1 -- 172.21.15.39:0/3942427139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc1043d0 con 0x7f22dc0fff80 2023-06-22T18:11:00.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.068+0000 7f22d2ffd700 1 -- 172.21.15.39:0/3942427139 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7f22cc042ad0 con 0x7f22dc0fff80 2023-06-22T18:11:00.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.070+0000 7f22d2ffd700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f22c809b680 0x7f22c809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.070+0000 7f22d2ffd700 1 -- 172.21.15.39:0/3942427139 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f22cc0c4710 con 0x7f22dc0fff80 2023-06-22T18:11:00.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.070+0000 7f22e219b700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f22c809b680 0x7f22c809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.071+0000 7f22e219b700 1 --2- 172.21.15.39:0/3942427139 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7f22c809b680 0x7f22c809db30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f22d80045a0 tx=0x7f22d8009e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:11:00.074 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.072+0000 7f22d2ffd700 1 -- 172.21.15.39:0/3942427139 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7f22cc090020 con 0x7f22dc0fff80 2023-06-22T18:11:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 1 closest matches: 2023-06-22T18:11:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-22T18:11:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2023-06-22T18:11:00.359 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-22T18:11:00.360 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-06-22T18:11:00.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-06-22T18:11:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 -- 172.21.15.39:0/3479181066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fdd00 msgr2=0x7fae7c0fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:11:00.561 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 --2- 172.21.15.39:0/3479181066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fdd00 0x7fae7c0fe150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0049c0 tx=0x7fae6c009e30 comp rx=0 tx=0).stop 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 -- 172.21.15.39:0/3479181066 shutdown_connections 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 --2- 172.21.15.39:0/3479181066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7c0fe690 0x7fae7c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 --2- 172.21.15.39:0/3479181066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae7c0fcaf0 0x7fae7c0fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 --2- 172.21.15.39:0/3479181066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fdd00 0x7fae7c0fe150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 -- 172.21.15.39:0/3479181066 >> 172.21.15.39:0/3479181066 conn(0x7fae7c0f8210 msgr2=0x7fae7c0fa680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 -- 172.21.15.39:0/3479181066 shutdown_connections 2023-06-22T18:11:00.562 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.558+0000 7fae8219d700 1 -- 172.21.15.39:0/3479181066 wait complete. 2023-06-22T18:11:00.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.559+0000 7fae8219d700 1 Processor -- start 2023-06-22T18:11:00.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.559+0000 7fae8219d700 1 -- start start 2023-06-22T18:11:00.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8219d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fcaf0 0x7fae7c10f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8219d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7c0fdd00 0x7fae7c10ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.564 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8219d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae7c0fe690 0x7fae7c116fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8219d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1175b0 con 0x7fae7c0fe690 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8219d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c117720 con 0x7fae7c0fcaf0 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8219d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1178e0 con 0x7fae7c0fdd00 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7c0fdd00 0x7fae7c10ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7c0fdd00 0x7fae7c10ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41370/0 (socket says 172.21.15.39:41370) 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 -- 172.21.15.39:0/178183588 learned_addr learned my addr 172.21.15.39:0/178183588 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8199c700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae7c0fe690 0x7fae7c116fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.565 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8119b700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fcaf0 0x7fae7c10f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 -- 172.21.15.39:0/178183588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fcaf0 msgr2=0x7fae7c10f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fae7c0fcaf0 0x7fae7c10f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 -- 172.21.15.39:0/178183588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae7c0fe690 msgr2=0x7fae7c116fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae7c0fe690 0x7fae7c116fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.560+0000 7fae8099a700 1 -- 172.21.15.39:0/178183588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c005040 con 0x7fae7c0fdd00 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.561+0000 7fae8099a700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7c0fdd00 0x7fae7c10ff10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009180 tx=0x7fae6c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.561+0000 7fae71ffb700 1 -- 172.21.15.39:0/178183588 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c020070 con 0x7fae7c0fdd00 2023-06-22T18:11:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.561+0000 7fae8219d700 1 -- 172.21.15.39:0/178183588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c117b60 con 0x7fae7c0fdd00 2023-06-22T18:11:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.561+0000 7fae71ffb700 1 -- 172.21.15.39:0/178183588 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae6c009520 con 0x7fae7c0fdd00 2023-06-22T18:11:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.561+0000 7fae71ffb700 1 -- 172.21.15.39:0/178183588 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00e070 con 0x7fae7c0fdd00 2023-06-22T18:11:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.561+0000 7fae8219d700 1 -- 172.21.15.39:0/178183588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c118040 con 0x7fae7c0fdd00 2023-06-22T18:11:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.562+0000 7fae8219d700 1 -- 172.21.15.39:0/178183588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c10a350 con 0x7fae7c0fdd00 2023-06-22T18:11:00.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.564+0000 7fae71ffb700 1 -- 172.21.15.39:0/178183588 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 129201+0+0 (secure 0 0 0) 0x7fae6c009690 con 0x7fae7c0fdd00 2023-06-22T18:11:00.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.565+0000 7fae71ffb700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fae6809b680 0x7fae6809db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-22T18:11:00.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.565+0000 7fae71ffb700 1 -- 172.21.15.39:0/178183588 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fae6c0c5e60 con 0x7fae7c0fdd00 2023-06-22T18:11:00.568 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.565+0000 7fae8119b700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fae6809b680 0x7fae6809db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-22T18:11:00.569 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.566+0000 7fae8119b700 1 --2- 172.21.15.39:0/178183588 >> [v2:172.21.15.39:6800/2143792276,v1:172.21.15.39:6801/2143792276] conn(0x7fae6809b680 0x7fae6809db30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae78009b80 tx=0x7fae7800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-22T18:11:00.571 INFO:teuthology.orchestra.run.smithi039.stderr:2023-06-22T18:11:00.569+0000 7fae71ffb700 1 -- 172.21.15.39:0/178183588 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174561 (secure 0 0 0) 0x7fae6c091770 con 0x7fae7c0fdd00 2023-06-22T18:11:00.602 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/1541002894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: pgmap v287: 97 pgs: 4 creating+peering, 34 active+clean, 59 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 0 op/s 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: mds.? [v2:172.21.15.101:6838/3279622767,v1:172.21.15.101:6839/3279622767] up:active 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: from='client.? 172.21.15.39:0/2069292772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: osdmap e85: 12 total, 12 up, 12 in 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.603 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.604 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:00 smithi101 ceph-mon[116267]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/1541002894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:11:00.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: pgmap v287: 97 pgs: 4 creating+peering, 34 active+clean, 59 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 0 op/s 2023-06-22T18:11:00.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: mds.? [v2:172.21.15.101:6838/3279622767,v1:172.21.15.101:6839/3279622767] up:active 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: from='client.? 172.21.15.39:0/2069292772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: osdmap e85: 12 total, 12 up, 12 in 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.695 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:00 smithi039 ceph-mon[109976]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/1541002894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:11:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: pgmap v287: 97 pgs: 4 creating+peering, 34 active+clean, 59 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 0 op/s 2023-06-22T18:11:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: mds.? [v2:172.21.15.101:6838/3279622767,v1:172.21.15.101:6839/3279622767] up:active 2023-06-22T18:11:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-22T18:11:00.734 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: from='client.? 172.21.15.39:0/2069292772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-22T18:11:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: osdmap e85: 12 total, 12 up, 12 in 2023-06-22T18:11:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.735 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:00 smithi133 ceph-mon[116867]: from='mgr.14152 172.21.15.39:0/1778323993' entity='mgr.x' 2023-06-22T18:11:00.831 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 1 closest matches: 2023-06-22T18:11:00.831 INFO:teuthology.orchestra.run.smithi039.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-22T18:11:00.832 INFO:teuthology.orchestra.run.smithi039.stderr:Error EINVAL: invalid command 2023-06-22T18:11:00.839 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-22T18:11:00.840 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 896, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 714, in create self.set_max_mds(max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 583, in set_max_mds self.set_var("max_mds", "%d" % max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5' 2023-06-22T18:11:00.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:11:00.908 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:11:00.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:11:00.970 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-06-22T18:11:00.971 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-22T18:11:01.009 DEBUG:teuthology.orchestra.run.smithi101:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-22T18:11:01.025 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-22T18:11:01.041 INFO:tasks.cephadm:Stopping all daemons... 2023-06-22T18:11:01.041 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-06-22T18:11:01.042 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a 2023-06-22T18:11:01.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:01 smithi039 systemd[1]: Stopping Ceph mon.a for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:01.895 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:01 smithi039 conmon[109951]: 2023-06-22T18:11:01.578+0000 7f815a5b5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:01.895 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:01 smithi039 conmon[109951]: 2023-06-22T18:11:01.578+0000 7f815a5b5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-06-22T18:11:02.149 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jun 22 18:11:01 smithi039 bash[156570]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mon-a 2023-06-22T18:11:02.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.a.service' 2023-06-22T18:11:02.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:02.574 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-06-22T18:11:02.575 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-06-22T18:11:02.575 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.b 2023-06-22T18:11:02.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:02 smithi101 systemd[1]: Stopping Ceph mon.b for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:03.482 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:03 smithi101 conmon[116243]: 2023-06-22T18:11:03.126+0000 7f5f10ffc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:03.482 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:03 smithi101 conmon[116243]: 2023-06-22T18:11:03.126+0000 7f5f10ffc700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-06-22T18:11:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jun 22 18:11:03 smithi101 bash[150885]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mon-b 2023-06-22T18:11:04.102 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:11:03 smithi101 conmon[148913]: 2023-06-22T18:11:03.819+0000 7fc4bbbe4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-22T18:11:04.395 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.b.service' 2023-06-22T18:11:04.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:04.464 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-06-22T18:11:04.464 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-06-22T18:11:04.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.c 2023-06-22T18:11:04.783 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:04 smithi133 systemd[1]: Stopping Ceph mon.c for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:04.783 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:04 smithi133 conmon[116824]: 2023-06-22T18:11:04.777+0000 7f423f777700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:04.784 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:04 smithi133 conmon[116824]: 2023-06-22T18:11:04.777+0000 7f423f777700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-06-22T18:11:05.484 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jun 22 18:11:05 smithi133 bash[153453]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mon-c 2023-06-22T18:11:05.885 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mon.c.service' 2023-06-22T18:11:05.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:05.933 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-06-22T18:11:05.934 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-06-22T18:11:05.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x 2023-06-22T18:11:06.444 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:11:06 smithi039 systemd[1]: Stopping Ceph mgr.x for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:06.788 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jun 22 18:11:06 smithi039 bash[157587]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mgr-x 2023-06-22T18:11:07.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.x.service' 2023-06-22T18:11:07.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:07.320 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-06-22T18:11:07.321 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-06-22T18:11:07.321 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.y 2023-06-22T18:11:07.603 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jun 22 18:11:07 smithi101 systemd[1]: Stopping Ceph mgr.y for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:08.468 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.y.service' 2023-06-22T18:11:08.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:08.504 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-06-22T18:11:08.505 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-06-22T18:11:08.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.z 2023-06-22T18:11:08.950 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:11:08 smithi133 systemd[1]: Stopping Ceph mgr.z for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:09.550 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.z.service' 2023-06-22T18:11:09.572 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:11:09 smithi133 bash[153745]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mgr-z 2023-06-22T18:11:09.572 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:11:09 smithi133 systemd[1]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-06-22T18:11:09.572 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:11:09 smithi133 systemd[1]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mgr.z.service: Failed with result 'exit-code'. 2023-06-22T18:11:09.573 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jun 22 18:11:09 smithi133 systemd[1]: Stopped Ceph mgr.z for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:11:09.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:09.585 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-06-22T18:11:09.585 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-06-22T18:11:09.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.0 2023-06-22T18:11:09.944 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:11:09 smithi039 systemd[1]: Stopping Ceph osd.0 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:10.444 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:11:10 smithi039 conmon[126227]: 2023-06-22T18:11:10.110+0000 7fdb114b2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:10.444 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:11:10 smithi039 conmon[126227]: 2023-06-22T18:11:10.110+0000 7fdb114b2700 -1 osd.0 85 *** Got signal Terminated *** 2023-06-22T18:11:10.445 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:11:10 smithi039 conmon[126227]: 2023-06-22T18:11:10.110+0000 7fdb114b2700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:15.944 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jun 22 18:11:15 smithi039 bash[157871]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-0 2023-06-22T18:11:17.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.0.service' 2023-06-22T18:11:17.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:17.880 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-06-22T18:11:17.880 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-06-22T18:11:17.881 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.1 2023-06-22T18:11:18.323 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:11:17 smithi101 systemd[1]: Stopping Ceph osd.1 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:18.602 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:11:18 smithi101 conmon[122684]: 2023-06-22T18:11:18.318+0000 7fcf12b07700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:18.603 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:11:18 smithi101 conmon[122684]: 2023-06-22T18:11:18.318+0000 7fcf12b07700 -1 osd.1 85 *** Got signal Terminated *** 2023-06-22T18:11:18.603 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:11:18 smithi101 conmon[122684]: 2023-06-22T18:11:18.318+0000 7fcf12b07700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:24.602 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jun 22 18:11:24 smithi101 bash[151813]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-1 2023-06-22T18:11:25.838 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.1.service' 2023-06-22T18:11:25.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:25.875 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-06-22T18:11:25.876 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-06-22T18:11:25.877 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.2 2023-06-22T18:11:26.231 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:11:25 smithi133 systemd[1]: Stopping Ceph osd.2 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:26.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:11:26 smithi133 conmon[124549]: 2023-06-22T18:11:26.227+0000 7f504136f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:26.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:11:26 smithi133 conmon[124549]: 2023-06-22T18:11:26.227+0000 7f504136f700 -1 osd.2 85 *** Got signal Terminated *** 2023-06-22T18:11:26.484 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:11:26 smithi133 conmon[124549]: 2023-06-22T18:11:26.227+0000 7f504136f700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:32.234 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jun 22 18:11:31 smithi133 bash[154035]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-2 2023-06-22T18:11:33.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.2.service' 2023-06-22T18:11:33.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:33.676 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-06-22T18:11:33.676 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-06-22T18:11:33.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.3 2023-06-22T18:11:34.194 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:11:33 smithi039 systemd[1]: Stopping Ceph osd.3 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:34.694 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:11:34 smithi039 conmon[133234]: 2023-06-22T18:11:34.197+0000 7f4a3d290700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:34.694 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:11:34 smithi039 conmon[133234]: 2023-06-22T18:11:34.197+0000 7f4a3d290700 -1 osd.3 85 *** Got signal Terminated *** 2023-06-22T18:11:34.694 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:11:34 smithi039 conmon[133234]: 2023-06-22T18:11:34.197+0000 7f4a3d290700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:37.734 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:37 smithi133 conmon[145608]: 2023-06-22T18:11:37.325+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:38.733 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:38 smithi133 conmon[138513]: 2023-06-22T18:11:38.308+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:38.734 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:38 smithi133 conmon[145608]: 2023-06-22T18:11:38.359+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:38.734 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:38 smithi133 conmon[131441]: 2023-06-22T18:11:38.554+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:39.194 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:11:38 smithi039 conmon[133234]: 2023-06-22T18:11:38.840+0000 7f4a3788a700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.919442+0000 front 2023-06-22T18:11:13.919475+0000 (oldest deadline 2023-06-22T18:11:38.619597+0000) 2023-06-22T18:11:39.734 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:39 smithi133 conmon[131441]: 2023-06-22T18:11:39.591+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:39.734 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:39 smithi133 conmon[138513]: 2023-06-22T18:11:39.356+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:39.734 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:39 smithi133 conmon[145608]: 2023-06-22T18:11:39.382+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:39.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:39 smithi101 conmon[143689]: 2023-06-22T18:11:39.558+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:40.194 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jun 22 18:11:39 smithi039 bash[158430]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-3 2023-06-22T18:11:40.612 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:40 smithi133 conmon[138513]: 2023-06-22T18:11:40.354+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:40.613 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:40 smithi133 conmon[145608]: 2023-06-22T18:11:40.393+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:40.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:40 smithi101 conmon[143689]: 2023-06-22T18:11:40.536+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:40.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:40 smithi133 conmon[131441]: 2023-06-22T18:11:40.609+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:41.626 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:41 smithi039 conmon[147329]: 2023-06-22T18:11:41.439+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:41.734 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:41 smithi133 conmon[138513]: 2023-06-22T18:11:41.372+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:41.734 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:41 smithi133 conmon[131441]: 2023-06-22T18:11:41.603+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:41.735 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:41 smithi133 conmon[145608]: 2023-06-22T18:11:41.423+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:41.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.3.service' 2023-06-22T18:11:41.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:41.847 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-06-22T18:11:41.848 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-06-22T18:11:41.848 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.4 2023-06-22T18:11:41.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:41 smithi101 conmon[143689]: 2023-06-22T18:11:41.543+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:41.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:41 smithi039 conmon[140288]: 2023-06-22T18:11:41.623+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:42.201 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:11:41 smithi101 systemd[1]: Stopping Ceph osd.4 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:42.596 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:11:42 smithi101 conmon[129640]: 2023-06-22T18:11:42.198+0000 7f47e7d0b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:42.596 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:11:42 smithi101 conmon[129640]: 2023-06-22T18:11:42.198+0000 7f47e7d0b700 -1 osd.4 85 *** Got signal Terminated *** 2023-06-22T18:11:42.596 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:11:42 smithi101 conmon[129640]: 2023-06-22T18:11:42.198+0000 7f47e7d0b700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:42.641 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:42 smithi133 conmon[138513]: 2023-06-22T18:11:42.358+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:42.641 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:42 smithi133 conmon[145608]: 2023-06-22T18:11:42.420+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:42.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:42 smithi101 conmon[143689]: 2023-06-22T18:11:42.593+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:42.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:42 smithi039 conmon[140288]: 2023-06-22T18:11:42.656+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:42.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:42 smithi039 conmon[147329]: 2023-06-22T18:11:42.473+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:42.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:42 smithi133 conmon[131441]: 2023-06-22T18:11:42.638+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:43.673 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:43 smithi133 conmon[138513]: 2023-06-22T18:11:43.381+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:43.673 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:43 smithi133 conmon[145608]: 2023-06-22T18:11:43.432+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:43.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:43 smithi101 conmon[143689]: 2023-06-22T18:11:43.560+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:43.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:43 smithi039 conmon[140288]: 2023-06-22T18:11:43.664+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:43.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:43 smithi039 conmon[147329]: 2023-06-22T18:11:43.502+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:43.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:43 smithi133 conmon[131441]: 2023-06-22T18:11:43.670+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:44.657 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:44 smithi133 conmon[138513]: 2023-06-22T18:11:44.416+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:44.658 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:44 smithi133 conmon[145608]: 2023-06-22T18:11:44.392+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:44.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:44 smithi101 conmon[143689]: 2023-06-22T18:11:44.585+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:44.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:44 smithi039 conmon[140288]: 2023-06-22T18:11:44.682+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:44.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:44 smithi039 conmon[147329]: 2023-06-22T18:11:44.528+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:44.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:44 smithi133 conmon[131441]: 2023-06-22T18:11:44.655+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:45.699 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:45 smithi133 conmon[138513]: 2023-06-22T18:11:45.404+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:45.700 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:45 smithi133 conmon[145608]: 2023-06-22T18:11:45.358+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:45.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:45 smithi101 conmon[143689]: 2023-06-22T18:11:45.547+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:45.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:45 smithi039 conmon[140288]: 2023-06-22T18:11:45.638+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:45.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:45 smithi039 conmon[140288]: 2023-06-22T18:11:45.638+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:45.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:45 smithi039 conmon[147329]: 2023-06-22T18:11:45.526+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:45.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:45 smithi133 conmon[131441]: 2023-06-22T18:11:45.696+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:45.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:45 smithi133 conmon[131441]: 2023-06-22T18:11:45.696+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:46.653 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:46 smithi133 conmon[138513]: 2023-06-22T18:11:46.454+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:46.654 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:46 smithi133 conmon[145608]: 2023-06-22T18:11:46.317+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:46.654 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:46 smithi133 conmon[145608]: 2023-06-22T18:11:46.317+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:46.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:46 smithi101 conmon[143689]: 2023-06-22T18:11:46.512+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:46.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:46 smithi039 conmon[140288]: 2023-06-22T18:11:46.609+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:46.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:46 smithi039 conmon[140288]: 2023-06-22T18:11:46.609+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:46.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:46 smithi039 conmon[147329]: 2023-06-22T18:11:46.563+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:46.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:46 smithi039 conmon[147329]: 2023-06-22T18:11:46.563+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:46.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:46 smithi133 conmon[131441]: 2023-06-22T18:11:46.651+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:46.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:46 smithi133 conmon[131441]: 2023-06-22T18:11:46.651+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:47.352 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:11:46 smithi101 conmon[129640]: 2023-06-22T18:11:46.962+0000 7f47e2305700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.503836+0000 front 2023-06-22T18:11:20.503880+0000 (oldest deadline 2023-06-22T18:11:46.403963+0000) 2023-06-22T18:11:47.353 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:47 smithi101 conmon[136691]: 2023-06-22T18:11:47.104+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:47.657 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:47 smithi133 conmon[138513]: 2023-06-22T18:11:47.498+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:47.657 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:47 smithi133 conmon[145608]: 2023-06-22T18:11:47.318+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:47.658 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:47 smithi133 conmon[145608]: 2023-06-22T18:11:47.318+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:47.725 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:47 smithi101 conmon[143689]: 2023-06-22T18:11:47.470+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:47.725 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:47 smithi101 conmon[143689]: 2023-06-22T18:11:47.470+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:47.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:47 smithi039 conmon[140288]: 2023-06-22T18:11:47.648+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:47.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:47 smithi039 conmon[140288]: 2023-06-22T18:11:47.648+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:47.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:47 smithi039 conmon[147329]: 2023-06-22T18:11:47.613+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:47.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:47 smithi039 conmon[147329]: 2023-06-22T18:11:47.613+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:47.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:47 smithi133 conmon[131441]: 2023-06-22T18:11:47.654+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:47.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:47 smithi133 conmon[131441]: 2023-06-22T18:11:47.654+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:48.352 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jun 22 18:11:48 smithi101 bash[152354]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-4 2023-06-22T18:11:48.353 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:48 smithi101 conmon[136691]: 2023-06-22T18:11:48.141+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:48.683 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:48 smithi133 conmon[138513]: 2023-06-22T18:11:48.498+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:48.684 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:48 smithi133 conmon[138513]: 2023-06-22T18:11:48.498+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:48.684 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:48 smithi133 conmon[145608]: 2023-06-22T18:11:48.294+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:48.685 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:48 smithi133 conmon[145608]: 2023-06-22T18:11:48.294+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:48.721 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:48 smithi101 conmon[143689]: 2023-06-22T18:11:48.453+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:48.722 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:48 smithi101 conmon[143689]: 2023-06-22T18:11:48.453+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:48.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:48 smithi039 conmon[140288]: 2023-06-22T18:11:48.674+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:48.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:48 smithi039 conmon[140288]: 2023-06-22T18:11:48.674+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:48.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:48 smithi039 conmon[147329]: 2023-06-22T18:11:48.590+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:48.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:48 smithi039 conmon[147329]: 2023-06-22T18:11:48.590+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:48.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:48 smithi133 conmon[131441]: 2023-06-22T18:11:48.681+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:48.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:48 smithi133 conmon[131441]: 2023-06-22T18:11:48.681+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:49.352 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:49 smithi101 conmon[136691]: 2023-06-22T18:11:49.191+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:49.599 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.4.service' 2023-06-22T18:11:49.665 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:49 smithi133 conmon[138513]: 2023-06-22T18:11:49.523+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:49.666 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:49 smithi133 conmon[138513]: 2023-06-22T18:11:49.523+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:49.666 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:49 smithi133 conmon[145608]: 2023-06-22T18:11:49.295+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:49.666 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:49 smithi133 conmon[145608]: 2023-06-22T18:11:49.295+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:49.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:49.689 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-06-22T18:11:49.689 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-06-22T18:11:49.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.5 2023-06-22T18:11:49.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:49 smithi101 conmon[143689]: 2023-06-22T18:11:49.495+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:49.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:49 smithi101 conmon[143689]: 2023-06-22T18:11:49.495+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:49.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:49 smithi039 conmon[140288]: 2023-06-22T18:11:49.680+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:49.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:49 smithi039 conmon[140288]: 2023-06-22T18:11:49.680+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:49.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:49 smithi039 conmon[147329]: 2023-06-22T18:11:49.544+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:49.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:49 smithi039 conmon[147329]: 2023-06-22T18:11:49.544+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:49.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:49 smithi133 conmon[131441]: 2023-06-22T18:11:49.661+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:49.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:49 smithi133 conmon[131441]: 2023-06-22T18:11:49.661+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:49.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:49 smithi133 systemd[1]: Stopping Ceph osd.5 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:50.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[145608]: 2023-06-22T18:11:50.269+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:50.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[145608]: 2023-06-22T18:11:50.269+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:50.485 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[131441]: 2023-06-22T18:11:50.047+0000 7f2bed142700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:50.485 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[131441]: 2023-06-22T18:11:50.047+0000 7f2bed142700 -1 osd.5 85 *** Got signal Terminated *** 2023-06-22T18:11:50.485 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[131441]: 2023-06-22T18:11:50.047+0000 7f2bed142700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:50.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:50 smithi101 conmon[136691]: 2023-06-22T18:11:50.227+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:50.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:50 smithi101 conmon[143689]: 2023-06-22T18:11:50.446+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:50.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:50 smithi101 conmon[143689]: 2023-06-22T18:11:50.446+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:50.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:50 smithi039 conmon[140288]: 2023-06-22T18:11:50.713+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:50.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:50 smithi039 conmon[140288]: 2023-06-22T18:11:50.713+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:50.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:50 smithi039 conmon[147329]: 2023-06-22T18:11:50.495+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:50.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:50 smithi039 conmon[147329]: 2023-06-22T18:11:50.495+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:50.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[138513]: 2023-06-22T18:11:50.542+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:50.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[138513]: 2023-06-22T18:11:50.542+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:50.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[131441]: 2023-06-22T18:11:50.613+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:50.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:50 smithi133 conmon[131441]: 2023-06-22T18:11:50.613+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:51.569 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:51 smithi133 conmon[138513]: 2023-06-22T18:11:51.496+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:51.570 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:51 smithi133 conmon[138513]: 2023-06-22T18:11:51.496+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:51.570 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:51 smithi133 conmon[145608]: 2023-06-22T18:11:51.257+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:51.570 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:51 smithi133 conmon[145608]: 2023-06-22T18:11:51.257+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:51.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:51 smithi101 conmon[136691]: 2023-06-22T18:11:51.225+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:51.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:51 smithi101 conmon[143689]: 2023-06-22T18:11:51.443+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:51.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:51 smithi101 conmon[143689]: 2023-06-22T18:11:51.443+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:51.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:51 smithi039 conmon[140288]: 2023-06-22T18:11:51.665+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:51.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:51 smithi039 conmon[140288]: 2023-06-22T18:11:51.665+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:51.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:51 smithi039 conmon[147329]: 2023-06-22T18:11:51.486+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:51.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:51 smithi039 conmon[147329]: 2023-06-22T18:11:51.486+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:51.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:51 smithi133 conmon[131441]: 2023-06-22T18:11:51.565+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:51.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:51 smithi133 conmon[131441]: 2023-06-22T18:11:51.566+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:52.537 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:52 smithi133 conmon[145608]: 2023-06-22T18:11:52.263+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:52.537 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:52 smithi133 conmon[145608]: 2023-06-22T18:11:52.263+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:52.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:52 smithi101 conmon[136691]: 2023-06-22T18:11:52.247+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:52.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:52 smithi101 conmon[143689]: 2023-06-22T18:11:52.427+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:52.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:52 smithi101 conmon[143689]: 2023-06-22T18:11:52.427+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:52.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:52 smithi039 conmon[140288]: 2023-06-22T18:11:52.679+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:52.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:52 smithi039 conmon[140288]: 2023-06-22T18:11:52.679+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:52.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:52 smithi039 conmon[147329]: 2023-06-22T18:11:52.478+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:52.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:52 smithi039 conmon[147329]: 2023-06-22T18:11:52.478+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:52.983 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:52 smithi133 conmon[131441]: 2023-06-22T18:11:52.570+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:52.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:52 smithi133 conmon[131441]: 2023-06-22T18:11:52.570+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:52.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:52 smithi133 conmon[138513]: 2023-06-22T18:11:52.534+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:52.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:52 smithi133 conmon[138513]: 2023-06-22T18:11:52.534+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:53.535 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[145608]: 2023-06-22T18:11:53.267+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:53.536 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[145608]: 2023-06-22T18:11:53.267+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:53.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:53 smithi101 conmon[136691]: 2023-06-22T18:11:53.248+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:53.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:53 smithi101 conmon[143689]: 2023-06-22T18:11:53.436+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:53.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:53 smithi101 conmon[143689]: 2023-06-22T18:11:53.436+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:53.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:53 smithi039 conmon[140288]: 2023-06-22T18:11:53.632+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:53.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:53 smithi039 conmon[140288]: 2023-06-22T18:11:53.632+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:53.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:53 smithi039 conmon[140288]: 2023-06-22T18:11:53.632+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:11:53.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:53 smithi039 conmon[147329]: 2023-06-22T18:11:53.495+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:53.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:53 smithi039 conmon[147329]: 2023-06-22T18:11:53.495+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:53.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[131441]: 2023-06-22T18:11:53.532+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:53.984 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[131441]: 2023-06-22T18:11:53.532+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:53.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[138513]: 2023-06-22T18:11:53.559+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:53.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[138513]: 2023-06-22T18:11:53.559+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:53.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:53 smithi133 conmon[138513]: 2023-06-22T18:11:53.559+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:11:54.600 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[131441]: 2023-06-22T18:11:54.512+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.560555+0000 front 2023-06-22T18:11:13.560756+0000 (oldest deadline 2023-06-22T18:11:38.260626+0000) 2023-06-22T18:11:54.600 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[131441]: 2023-06-22T18:11:54.512+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:19.962054+0000 front 2023-06-22T18:11:19.962043+0000 (oldest deadline 2023-06-22T18:11:44.662066+0000) 2023-06-22T18:11:54.600 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[131441]: 2023-06-22T18:11:54.512+0000 7f2bea742700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.063779+0000 front 2023-06-22T18:11:31.063654+0000 (oldest deadline 2023-06-22T18:11:53.963949+0000) 2023-06-22T18:11:54.600 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[145608]: 2023-06-22T18:11:54.310+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:54.601 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[145608]: 2023-06-22T18:11:54.310+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:54.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:54 smithi101 conmon[136691]: 2023-06-22T18:11:54.261+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:54.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:54 smithi101 conmon[136691]: 2023-06-22T18:11:54.261+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:11:54.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:54 smithi101 conmon[143689]: 2023-06-22T18:11:54.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:54.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:54 smithi101 conmon[143689]: 2023-06-22T18:11:54.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:54.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:54 smithi101 conmon[143689]: 2023-06-22T18:11:54.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:11:54.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:54 smithi039 conmon[140288]: 2023-06-22T18:11:54.631+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:54.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:54 smithi039 conmon[140288]: 2023-06-22T18:11:54.631+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:54.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:54 smithi039 conmon[140288]: 2023-06-22T18:11:54.631+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:11:54.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:54 smithi039 conmon[147329]: 2023-06-22T18:11:54.529+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:54.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:54 smithi039 conmon[147329]: 2023-06-22T18:11:54.529+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:54.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[138513]: 2023-06-22T18:11:54.596+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:54.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[138513]: 2023-06-22T18:11:54.596+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:54.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:54 smithi133 conmon[138513]: 2023-06-22T18:11:54.596+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:11:55.558 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:55 smithi133 conmon[145608]: 2023-06-22T18:11:55.276+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:55.559 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:55 smithi133 conmon[145608]: 2023-06-22T18:11:55.276+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:55.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:55 smithi101 conmon[136691]: 2023-06-22T18:11:55.238+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:55.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:55 smithi101 conmon[136691]: 2023-06-22T18:11:55.238+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:11:55.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:55 smithi101 conmon[143689]: 2023-06-22T18:11:55.378+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:55.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:55 smithi101 conmon[143689]: 2023-06-22T18:11:55.378+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:55.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:55 smithi101 conmon[143689]: 2023-06-22T18:11:55.378+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:11:55.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:55 smithi039 conmon[140288]: 2023-06-22T18:11:55.675+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:55.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:55 smithi039 conmon[140288]: 2023-06-22T18:11:55.675+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:55.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:55 smithi039 conmon[140288]: 2023-06-22T18:11:55.675+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:11:55.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:55 smithi039 conmon[147329]: 2023-06-22T18:11:55.562+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:55.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:55 smithi039 conmon[147329]: 2023-06-22T18:11:55.562+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:55.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:55 smithi039 conmon[147329]: 2023-06-22T18:11:55.562+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:11:55.945 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:55 smithi133 conmon[138513]: 2023-06-22T18:11:55.638+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:55.945 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:55 smithi133 conmon[138513]: 2023-06-22T18:11:55.638+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:55.945 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:55 smithi133 conmon[138513]: 2023-06-22T18:11:55.638+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:11:56.233 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jun 22 18:11:55 smithi133 bash[154581]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-5 2023-06-22T18:11:56.583 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:56 smithi133 conmon[145608]: 2023-06-22T18:11:56.294+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:56.584 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:56 smithi133 conmon[145608]: 2023-06-22T18:11:56.294+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:56.584 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:56 smithi133 conmon[145608]: 2023-06-22T18:11:56.294+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:11:56.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:56 smithi101 conmon[136691]: 2023-06-22T18:11:56.236+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:56.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:56 smithi101 conmon[136691]: 2023-06-22T18:11:56.236+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:11:56.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:56 smithi101 conmon[143689]: 2023-06-22T18:11:56.362+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:56.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:56 smithi101 conmon[143689]: 2023-06-22T18:11:56.362+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:56.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:56 smithi101 conmon[143689]: 2023-06-22T18:11:56.362+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:11:56.917 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:56 smithi133 conmon[138513]: 2023-06-22T18:11:56.657+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:56.918 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:56 smithi133 conmon[138513]: 2023-06-22T18:11:56.657+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:56.918 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:56 smithi133 conmon[138513]: 2023-06-22T18:11:56.657+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:11:56.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:56 smithi039 conmon[140288]: 2023-06-22T18:11:56.697+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:56.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:56 smithi039 conmon[140288]: 2023-06-22T18:11:56.697+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:56.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:56 smithi039 conmon[140288]: 2023-06-22T18:11:56.697+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:11:56.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:56 smithi039 conmon[147329]: 2023-06-22T18:11:56.559+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:56.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:56 smithi039 conmon[147329]: 2023-06-22T18:11:56.559+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:56.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:56 smithi039 conmon[147329]: 2023-06-22T18:11:56.559+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:11:57.539 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:57 smithi133 conmon[145608]: 2023-06-22T18:11:57.258+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:57.540 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:57 smithi133 conmon[145608]: 2023-06-22T18:11:57.258+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:57.540 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:57 smithi133 conmon[145608]: 2023-06-22T18:11:57.258+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:11:57.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.5.service' 2023-06-22T18:11:57.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:11:57.585 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-06-22T18:11:57.585 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-06-22T18:11:57.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.6 2023-06-22T18:11:57.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:57 smithi101 conmon[136691]: 2023-06-22T18:11:57.239+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:57.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:57 smithi101 conmon[136691]: 2023-06-22T18:11:57.239+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:11:57.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:57 smithi101 conmon[143689]: 2023-06-22T18:11:57.336+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:57.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:57 smithi101 conmon[143689]: 2023-06-22T18:11:57.336+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:57.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:57 smithi101 conmon[143689]: 2023-06-22T18:11:57.336+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:11:57.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:57 smithi039 conmon[147329]: 2023-06-22T18:11:57.529+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:57.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:57 smithi039 conmon[147329]: 2023-06-22T18:11:57.529+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:57.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:57 smithi039 conmon[147329]: 2023-06-22T18:11:57.529+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:11:57.945 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:57 smithi039 systemd[1]: Stopping Ceph osd.6 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:11:57.945 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:57 smithi039 conmon[140288]: 2023-06-22T18:11:57.724+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:57.945 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:57 smithi039 conmon[140288]: 2023-06-22T18:11:57.724+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:57.945 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:57 smithi039 conmon[140288]: 2023-06-22T18:11:57.724+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:11:57.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:57 smithi133 conmon[138513]: 2023-06-22T18:11:57.630+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:57.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:57 smithi133 conmon[138513]: 2023-06-22T18:11:57.630+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:57.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:57 smithi133 conmon[138513]: 2023-06-22T18:11:57.630+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:11:58.444 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[140288]: 2023-06-22T18:11:58.119+0000 7f2c75fa2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:11:58.444 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[140288]: 2023-06-22T18:11:58.119+0000 7f2c75fa2700 -1 osd.6 85 *** Got signal Terminated *** 2023-06-22T18:11:58.444 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[140288]: 2023-06-22T18:11:58.120+0000 7f2c75fa2700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:11:58.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:58 smithi101 conmon[136691]: 2023-06-22T18:11:58.287+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:58.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:58 smithi101 conmon[136691]: 2023-06-22T18:11:58.287+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:11:58.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:58 smithi101 conmon[143689]: 2023-06-22T18:11:58.333+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:58.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:58 smithi101 conmon[143689]: 2023-06-22T18:11:58.333+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:58.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:58 smithi101 conmon[143689]: 2023-06-22T18:11:58.333+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:11:58.659 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:58 smithi133 conmon[145608]: 2023-06-22T18:11:58.307+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:58.660 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:58 smithi133 conmon[145608]: 2023-06-22T18:11:58.307+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:58.660 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:58 smithi133 conmon[145608]: 2023-06-22T18:11:58.307+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:11:58.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[140288]: 2023-06-22T18:11:58.756+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:11:58.944 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[140288]: 2023-06-22T18:11:58.756+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:11:58.945 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[140288]: 2023-06-22T18:11:58.756+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:11:58.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[147329]: 2023-06-22T18:11:58.532+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:58.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[147329]: 2023-06-22T18:11:58.532+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:58.946 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:58 smithi039 conmon[147329]: 2023-06-22T18:11:58.532+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:11:58.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:58 smithi133 conmon[138513]: 2023-06-22T18:11:58.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:58.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:58 smithi133 conmon[138513]: 2023-06-22T18:11:58.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:58.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:58 smithi133 conmon[138513]: 2023-06-22T18:11:58.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:11:59.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:59 smithi101 conmon[136691]: 2023-06-22T18:11:59.269+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:11:59.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:11:59 smithi101 conmon[136691]: 2023-06-22T18:11:59.269+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:11:59.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:59 smithi101 conmon[143689]: 2023-06-22T18:11:59.353+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:11:59.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:59 smithi101 conmon[143689]: 2023-06-22T18:11:59.353+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:11:59.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:11:59 smithi101 conmon[143689]: 2023-06-22T18:11:59.353+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:11:59.709 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:59 smithi133 conmon[145608]: 2023-06-22T18:11:59.336+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:11:59.710 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:59 smithi133 conmon[145608]: 2023-06-22T18:11:59.336+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:11:59.710 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:11:59 smithi133 conmon[145608]: 2023-06-22T18:11:59.336+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:11:59.802 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:59 smithi039 conmon[147329]: 2023-06-22T18:11:59.496+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:11:59.802 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:59 smithi039 conmon[147329]: 2023-06-22T18:11:59.496+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:11:59.802 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:11:59 smithi039 conmon[147329]: 2023-06-22T18:11:59.496+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:11:59.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:59 smithi133 conmon[138513]: 2023-06-22T18:11:59.706+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:11:59.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:59 smithi133 conmon[138513]: 2023-06-22T18:11:59.706+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:11:59.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:11:59 smithi133 conmon[138513]: 2023-06-22T18:11:59.706+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:00.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:59 smithi039 conmon[140288]: 2023-06-22T18:11:59.799+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:12:00.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:59 smithi039 conmon[140288]: 2023-06-22T18:11:59.799+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:12:00.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:11:59 smithi039 conmon[140288]: 2023-06-22T18:11:59.799+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:12:00.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:00 smithi101 conmon[136691]: 2023-06-22T18:12:00.242+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:00.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:00 smithi101 conmon[136691]: 2023-06-22T18:12:00.242+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:00.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:00 smithi101 conmon[136691]: 2023-06-22T18:12:00.242+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:00.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:00 smithi101 conmon[143689]: 2023-06-22T18:12:00.348+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:00.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:00 smithi101 conmon[143689]: 2023-06-22T18:12:00.348+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:00.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:00 smithi101 conmon[143689]: 2023-06-22T18:12:00.348+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:00.715 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:00 smithi133 conmon[145608]: 2023-06-22T18:12:00.323+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:00.716 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:00 smithi133 conmon[145608]: 2023-06-22T18:12:00.323+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:00.716 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:00 smithi133 conmon[145608]: 2023-06-22T18:12:00.323+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:00.796 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[147329]: 2023-06-22T18:12:00.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:00.796 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[147329]: 2023-06-22T18:12:00.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:00.796 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[147329]: 2023-06-22T18:12:00.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:00.796 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[147329]: 2023-06-22T18:12:00.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:00.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:00 smithi133 conmon[138513]: 2023-06-22T18:12:00.711+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:00.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:00 smithi133 conmon[138513]: 2023-06-22T18:12:00.711+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:00.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:00 smithi133 conmon[138513]: 2023-06-22T18:12:00.711+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:01.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[140288]: 2023-06-22T18:12:00.792+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:12:01.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[140288]: 2023-06-22T18:12:00.792+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:12:01.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:00 smithi039 conmon[140288]: 2023-06-22T18:12:00.792+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:12:01.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[136691]: 2023-06-22T18:12:01.241+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:01.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[136691]: 2023-06-22T18:12:01.241+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:01.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[136691]: 2023-06-22T18:12:01.241+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:01.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[143689]: 2023-06-22T18:12:01.370+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:01.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[143689]: 2023-06-22T18:12:01.370+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:01.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[143689]: 2023-06-22T18:12:01.370+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:01.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:01 smithi101 conmon[143689]: 2023-06-22T18:12:01.370+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:01.679 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:01 smithi133 conmon[145608]: 2023-06-22T18:12:01.319+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:01.680 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:01 smithi133 conmon[145608]: 2023-06-22T18:12:01.319+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:01.680 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:01 smithi133 conmon[145608]: 2023-06-22T18:12:01.319+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:01.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[147329]: 2023-06-22T18:12:01.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:01.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[147329]: 2023-06-22T18:12:01.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:01.827 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[147329]: 2023-06-22T18:12:01.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:01.828 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[147329]: 2023-06-22T18:12:01.475+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:01.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:01 smithi133 conmon[138513]: 2023-06-22T18:12:01.677+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:01.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:01 smithi133 conmon[138513]: 2023-06-22T18:12:01.677+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:01.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:01 smithi133 conmon[138513]: 2023-06-22T18:12:01.677+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:02.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[140288]: 2023-06-22T18:12:01.823+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:12:02.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[140288]: 2023-06-22T18:12:01.823+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:12:02.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:01 smithi039 conmon[140288]: 2023-06-22T18:12:01.823+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:12:02.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[136691]: 2023-06-22T18:12:02.222+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:02.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[136691]: 2023-06-22T18:12:02.222+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:02.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[136691]: 2023-06-22T18:12:02.222+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:02.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[143689]: 2023-06-22T18:12:02.343+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:02.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[143689]: 2023-06-22T18:12:02.343+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:02.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[143689]: 2023-06-22T18:12:02.343+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:02.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:02 smithi101 conmon[143689]: 2023-06-22T18:12:02.343+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:02.655 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:02 smithi133 conmon[145608]: 2023-06-22T18:12:02.286+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:02.655 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:02 smithi133 conmon[145608]: 2023-06-22T18:12:02.286+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:02.655 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:02 smithi133 conmon[145608]: 2023-06-22T18:12:02.286+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:02.782 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[147329]: 2023-06-22T18:12:02.449+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:02.782 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[147329]: 2023-06-22T18:12:02.449+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:02.783 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[147329]: 2023-06-22T18:12:02.449+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:02.783 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[147329]: 2023-06-22T18:12:02.449+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:02.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:02 smithi133 conmon[138513]: 2023-06-22T18:12:02.652+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:02.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:02 smithi133 conmon[138513]: 2023-06-22T18:12:02.652+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:02.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:02 smithi133 conmon[138513]: 2023-06-22T18:12:02.652+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:03.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[140288]: 2023-06-22T18:12:02.778+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.103926+0000 front 2023-06-22T18:11:15.103962+0000 (oldest deadline 2023-06-22T18:11:41.004083+0000) 2023-06-22T18:12:03.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[140288]: 2023-06-22T18:12:02.778+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.705116+0000 front 2023-06-22T18:11:22.705158+0000 (oldest deadline 2023-06-22T18:11:45.605120+0000) 2023-06-22T18:12:03.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[140288]: 2023-06-22T18:12:02.778+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.306630+0000 front 2023-06-22T18:11:30.306655+0000 (oldest deadline 2023-06-22T18:11:53.206579+0000) 2023-06-22T18:12:03.194 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:02 smithi039 conmon[140288]: 2023-06-22T18:12:02.778+0000 7f2c725a0700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.208123+0000 front 2023-06-22T18:11:37.208018+0000 (oldest deadline 2023-06-22T18:12:02.508047+0000) 2023-06-22T18:12:03.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[136691]: 2023-06-22T18:12:03.203+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:03.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[136691]: 2023-06-22T18:12:03.203+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:03.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[136691]: 2023-06-22T18:12:03.204+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:03.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[143689]: 2023-06-22T18:12:03.334+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:03.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[143689]: 2023-06-22T18:12:03.334+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:03.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[143689]: 2023-06-22T18:12:03.334+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:03.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:03 smithi101 conmon[143689]: 2023-06-22T18:12:03.334+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:03.634 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:03 smithi133 conmon[145608]: 2023-06-22T18:12:03.269+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:03.635 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:03 smithi133 conmon[145608]: 2023-06-22T18:12:03.269+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:03.635 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:03 smithi133 conmon[145608]: 2023-06-22T18:12:03.269+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:03.819 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jun 22 18:12:03 smithi039 bash[158989]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-6 2023-06-22T18:12:03.819 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:03 smithi039 conmon[147329]: 2023-06-22T18:12:03.488+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:03.820 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:03 smithi039 conmon[147329]: 2023-06-22T18:12:03.488+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:03.820 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:03 smithi039 conmon[147329]: 2023-06-22T18:12:03.488+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:03.820 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:03 smithi039 conmon[147329]: 2023-06-22T18:12:03.488+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:03.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:03 smithi133 conmon[138513]: 2023-06-22T18:12:03.631+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:03.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:03 smithi133 conmon[138513]: 2023-06-22T18:12:03.631+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:03.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:03 smithi133 conmon[138513]: 2023-06-22T18:12:03.631+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:04.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[136691]: 2023-06-22T18:12:04.181+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:04.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[136691]: 2023-06-22T18:12:04.181+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:04.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[136691]: 2023-06-22T18:12:04.181+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:04.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[143689]: 2023-06-22T18:12:04.381+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:04.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[143689]: 2023-06-22T18:12:04.381+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:04.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[143689]: 2023-06-22T18:12:04.381+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:04.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:04 smithi101 conmon[143689]: 2023-06-22T18:12:04.381+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:04.674 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:04 smithi133 conmon[145608]: 2023-06-22T18:12:04.243+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:04.675 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:04 smithi133 conmon[145608]: 2023-06-22T18:12:04.243+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:04.675 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:04 smithi133 conmon[145608]: 2023-06-22T18:12:04.243+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:04.777 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:04 smithi039 conmon[147329]: 2023-06-22T18:12:04.500+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:04.777 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:04 smithi039 conmon[147329]: 2023-06-22T18:12:04.500+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:04.778 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:04 smithi039 conmon[147329]: 2023-06-22T18:12:04.500+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:04.778 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:04 smithi039 conmon[147329]: 2023-06-22T18:12:04.500+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:04.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:04 smithi133 conmon[138513]: 2023-06-22T18:12:04.671+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:04.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:04 smithi133 conmon[138513]: 2023-06-22T18:12:04.671+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:04.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:04 smithi133 conmon[138513]: 2023-06-22T18:12:04.671+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:05.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[145608]: 2023-06-22T18:12:05.211+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:05.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[145608]: 2023-06-22T18:12:05.211+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:05.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[145608]: 2023-06-22T18:12:05.211+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:05.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[136691]: 2023-06-22T18:12:05.199+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:05.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[136691]: 2023-06-22T18:12:05.199+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:05.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[136691]: 2023-06-22T18:12:05.199+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:05.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[143689]: 2023-06-22T18:12:05.345+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:05.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[143689]: 2023-06-22T18:12:05.345+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:05.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[143689]: 2023-06-22T18:12:05.345+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:05.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:05 smithi101 conmon[143689]: 2023-06-22T18:12:05.345+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:05.694 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:05 smithi039 conmon[147329]: 2023-06-22T18:12:05.478+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:05.694 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:05 smithi039 conmon[147329]: 2023-06-22T18:12:05.478+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:05.694 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:05 smithi039 conmon[147329]: 2023-06-22T18:12:05.478+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:05.694 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:05 smithi039 conmon[147329]: 2023-06-22T18:12:05.478+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:05.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.6.service' 2023-06-22T18:12:05.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:05.855 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-06-22T18:12:05.855 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-06-22T18:12:05.856 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.7 2023-06-22T18:12:05.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[138513]: 2023-06-22T18:12:05.659+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:05.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[138513]: 2023-06-22T18:12:05.659+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:05.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[138513]: 2023-06-22T18:12:05.659+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:05.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:05 smithi133 conmon[138513]: 2023-06-22T18:12:05.659+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:06.159 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:05 smithi101 systemd[1]: Stopping Ceph osd.7 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:06.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[145608]: 2023-06-22T18:12:06.165+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:06.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[145608]: 2023-06-22T18:12:06.165+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:06.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[145608]: 2023-06-22T18:12:06.165+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:06.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[136691]: 2023-06-22T18:12:06.155+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:06.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[136691]: 2023-06-22T18:12:06.155+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:06.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[136691]: 2023-06-22T18:12:06.155+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:06.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[136691]: 2023-06-22T18:12:06.210+0000 7f9c25bed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:06.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[136691]: 2023-06-22T18:12:06.210+0000 7f9c25bed700 -1 osd.7 85 *** Got signal Terminated *** 2023-06-22T18:12:06.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[136691]: 2023-06-22T18:12:06.210+0000 7f9c25bed700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:12:06.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[143689]: 2023-06-22T18:12:06.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:06.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[143689]: 2023-06-22T18:12:06.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:06.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[143689]: 2023-06-22T18:12:06.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:06.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:06 smithi101 conmon[143689]: 2023-06-22T18:12:06.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:06.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:06 smithi039 conmon[147329]: 2023-06-22T18:12:06.486+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:06.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:06 smithi039 conmon[147329]: 2023-06-22T18:12:06.486+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:06.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:06 smithi039 conmon[147329]: 2023-06-22T18:12:06.486+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:06.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:06 smithi039 conmon[147329]: 2023-06-22T18:12:06.486+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:06.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[138513]: 2023-06-22T18:12:06.615+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:06.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[138513]: 2023-06-22T18:12:06.615+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:06.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[138513]: 2023-06-22T18:12:06.615+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:06.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:06 smithi133 conmon[138513]: 2023-06-22T18:12:06.615+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:07.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[145608]: 2023-06-22T18:12:07.195+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:07.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[145608]: 2023-06-22T18:12:07.195+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:07.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[145608]: 2023-06-22T18:12:07.195+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:07.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[136691]: 2023-06-22T18:12:07.153+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:07.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[136691]: 2023-06-22T18:12:07.153+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:07.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[136691]: 2023-06-22T18:12:07.153+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:07.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[143689]: 2023-06-22T18:12:07.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:07.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[143689]: 2023-06-22T18:12:07.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:07.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[143689]: 2023-06-22T18:12:07.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:07.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:07 smithi101 conmon[143689]: 2023-06-22T18:12:07.340+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:07.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:07 smithi039 conmon[147329]: 2023-06-22T18:12:07.505+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:07.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:07 smithi039 conmon[147329]: 2023-06-22T18:12:07.505+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:07.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:07 smithi039 conmon[147329]: 2023-06-22T18:12:07.505+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:07.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:07 smithi039 conmon[147329]: 2023-06-22T18:12:07.505+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:07.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[138513]: 2023-06-22T18:12:07.611+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:07.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[138513]: 2023-06-22T18:12:07.611+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:07.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[138513]: 2023-06-22T18:12:07.611+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:07.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:07 smithi133 conmon[138513]: 2023-06-22T18:12:07.611+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:08.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[145608]: 2023-06-22T18:12:08.177+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:08.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[145608]: 2023-06-22T18:12:08.177+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:08.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[145608]: 2023-06-22T18:12:08.177+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:08.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[136691]: 2023-06-22T18:12:08.156+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:08.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[136691]: 2023-06-22T18:12:08.156+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:08.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[136691]: 2023-06-22T18:12:08.156+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:08.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[143689]: 2023-06-22T18:12:08.332+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:08.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[143689]: 2023-06-22T18:12:08.332+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:08.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[143689]: 2023-06-22T18:12:08.332+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:08.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:08 smithi101 conmon[143689]: 2023-06-22T18:12:08.332+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:08.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:08 smithi039 conmon[147329]: 2023-06-22T18:12:08.535+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:08.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:08 smithi039 conmon[147329]: 2023-06-22T18:12:08.535+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:08.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:08 smithi039 conmon[147329]: 2023-06-22T18:12:08.535+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:08.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:08 smithi039 conmon[147329]: 2023-06-22T18:12:08.535+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:08.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:08 smithi039 conmon[147329]: 2023-06-22T18:12:08.535+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:08.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[138513]: 2023-06-22T18:12:08.618+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:08.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[138513]: 2023-06-22T18:12:08.618+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:08.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[138513]: 2023-06-22T18:12:08.618+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:08.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:08 smithi133 conmon[138513]: 2023-06-22T18:12:08.618+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:09.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[145608]: 2023-06-22T18:12:09.216+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:09.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[145608]: 2023-06-22T18:12:09.216+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:09.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[145608]: 2023-06-22T18:12:09.216+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:09.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[145608]: 2023-06-22T18:12:09.216+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:09.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[136691]: 2023-06-22T18:12:09.207+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:09.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[136691]: 2023-06-22T18:12:09.207+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:09.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[136691]: 2023-06-22T18:12:09.207+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:09.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[143689]: 2023-06-22T18:12:09.351+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:09.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[143689]: 2023-06-22T18:12:09.351+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:09.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[143689]: 2023-06-22T18:12:09.351+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:09.605 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:09 smithi101 conmon[143689]: 2023-06-22T18:12:09.351+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:09.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:09 smithi039 conmon[147329]: 2023-06-22T18:12:09.531+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:09.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:09 smithi039 conmon[147329]: 2023-06-22T18:12:09.531+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:09.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:09 smithi039 conmon[147329]: 2023-06-22T18:12:09.531+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:09.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:09 smithi039 conmon[147329]: 2023-06-22T18:12:09.531+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:09.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:09 smithi039 conmon[147329]: 2023-06-22T18:12:09.531+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:09.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[138513]: 2023-06-22T18:12:09.616+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:10.027 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[138513]: 2023-06-22T18:12:09.616+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:10.027 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[138513]: 2023-06-22T18:12:09.616+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:10.028 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[138513]: 2023-06-22T18:12:09.616+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:10.028 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:09 smithi133 conmon[138513]: 2023-06-22T18:12:09.616+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:10.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[145608]: 2023-06-22T18:12:10.189+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:10.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[145608]: 2023-06-22T18:12:10.189+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:10.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[145608]: 2023-06-22T18:12:10.189+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:10.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[145608]: 2023-06-22T18:12:10.189+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:10.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[136691]: 2023-06-22T18:12:10.166+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:10.602 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[136691]: 2023-06-22T18:12:10.166+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:10.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[136691]: 2023-06-22T18:12:10.166+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:10.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[136691]: 2023-06-22T18:12:10.166+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.672462+0000 front 2023-06-22T18:11:44.672332+0000 (oldest deadline 2023-06-22T18:12:09.372463+0000) 2023-06-22T18:12:10.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[143689]: 2023-06-22T18:12:10.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:10.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[143689]: 2023-06-22T18:12:10.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:10.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[143689]: 2023-06-22T18:12:10.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:10.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[143689]: 2023-06-22T18:12:10.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:10.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:10 smithi101 conmon[143689]: 2023-06-22T18:12:10.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:10.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:10 smithi039 conmon[147329]: 2023-06-22T18:12:10.577+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:10.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:10 smithi039 conmon[147329]: 2023-06-22T18:12:10.577+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:10.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:10 smithi039 conmon[147329]: 2023-06-22T18:12:10.577+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:10.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:10 smithi039 conmon[147329]: 2023-06-22T18:12:10.577+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:10.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:10 smithi039 conmon[147329]: 2023-06-22T18:12:10.577+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:10.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[138513]: 2023-06-22T18:12:10.604+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:10.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[138513]: 2023-06-22T18:12:10.604+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:10.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[138513]: 2023-06-22T18:12:10.604+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:10.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[138513]: 2023-06-22T18:12:10.604+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:10.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:10 smithi133 conmon[138513]: 2023-06-22T18:12:10.604+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:11.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[145608]: 2023-06-22T18:12:11.139+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:11.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[145608]: 2023-06-22T18:12:11.139+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:11.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[145608]: 2023-06-22T18:12:11.139+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:11.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[145608]: 2023-06-22T18:12:11.139+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:11.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[136691]: 2023-06-22T18:12:11.173+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.969533+0000 front 2023-06-22T18:11:22.969568+0000 (oldest deadline 2023-06-22T18:11:47.069728+0000) 2023-06-22T18:12:11.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[136691]: 2023-06-22T18:12:11.173+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:30.570914+0000 front 2023-06-22T18:11:30.570991+0000 (oldest deadline 2023-06-22T18:11:54.071016+0000) 2023-06-22T18:12:11.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[136691]: 2023-06-22T18:12:11.173+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:34.071569+0000 front 2023-06-22T18:11:34.071555+0000 (oldest deadline 2023-06-22T18:11:59.971440+0000) 2023-06-22T18:12:11.603 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[136691]: 2023-06-22T18:12:11.173+0000 7f9c201e7700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.672462+0000 front 2023-06-22T18:11:44.672332+0000 (oldest deadline 2023-06-22T18:12:09.372463+0000) 2023-06-22T18:12:11.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[143689]: 2023-06-22T18:12:11.330+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:11.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[143689]: 2023-06-22T18:12:11.330+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:11.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[143689]: 2023-06-22T18:12:11.330+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:11.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[143689]: 2023-06-22T18:12:11.330+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:11.605 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:11 smithi101 conmon[143689]: 2023-06-22T18:12:11.330+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:11.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:11 smithi039 conmon[147329]: 2023-06-22T18:12:11.618+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:11.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:11 smithi039 conmon[147329]: 2023-06-22T18:12:11.618+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:11.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:11 smithi039 conmon[147329]: 2023-06-22T18:12:11.618+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:11.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:11 smithi039 conmon[147329]: 2023-06-22T18:12:11.618+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:11.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:11 smithi039 conmon[147329]: 2023-06-22T18:12:11.618+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:11.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[138513]: 2023-06-22T18:12:11.614+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:11.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[138513]: 2023-06-22T18:12:11.614+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:11.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[138513]: 2023-06-22T18:12:11.614+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:11.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[138513]: 2023-06-22T18:12:11.614+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:11.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:11 smithi133 conmon[138513]: 2023-06-22T18:12:11.614+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:12.103 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jun 22 18:12:11 smithi101 bash[152899]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-7 2023-06-22T18:12:12.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[145608]: 2023-06-22T18:12:12.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:12.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[145608]: 2023-06-22T18:12:12.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:12.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[145608]: 2023-06-22T18:12:12.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:12.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[145608]: 2023-06-22T18:12:12.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:12.627 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:12 smithi101 conmon[143689]: 2023-06-22T18:12:12.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:12.627 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:12 smithi101 conmon[143689]: 2023-06-22T18:12:12.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:12.627 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:12 smithi101 conmon[143689]: 2023-06-22T18:12:12.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:12.627 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:12 smithi101 conmon[143689]: 2023-06-22T18:12:12.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:12.627 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:12 smithi101 conmon[143689]: 2023-06-22T18:12:12.361+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:12.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:12 smithi039 conmon[147329]: 2023-06-22T18:12:12.660+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:12.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:12 smithi039 conmon[147329]: 2023-06-22T18:12:12.660+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:12.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:12 smithi039 conmon[147329]: 2023-06-22T18:12:12.660+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:12.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:12 smithi039 conmon[147329]: 2023-06-22T18:12:12.660+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:12.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:12 smithi039 conmon[147329]: 2023-06-22T18:12:12.660+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:12.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[138513]: 2023-06-22T18:12:12.660+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:12.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[138513]: 2023-06-22T18:12:12.660+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:12.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[138513]: 2023-06-22T18:12:12.660+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:12.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[138513]: 2023-06-22T18:12:12.660+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:12.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:12 smithi133 conmon[138513]: 2023-06-22T18:12:12.660+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:13.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[145608]: 2023-06-22T18:12:13.094+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:13.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[145608]: 2023-06-22T18:12:13.094+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:13.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[145608]: 2023-06-22T18:12:13.094+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:13.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[145608]: 2023-06-22T18:12:13.094+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:13.525 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.7.service' 2023-06-22T18:12:13.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:13.565 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-06-22T18:12:13.566 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-06-22T18:12:13.566 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.8 2023-06-22T18:12:13.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:13 smithi101 conmon[143689]: 2023-06-22T18:12:13.407+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:13.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:13 smithi101 conmon[143689]: 2023-06-22T18:12:13.407+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:13.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:13 smithi101 conmon[143689]: 2023-06-22T18:12:13.407+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:13.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:13 smithi101 conmon[143689]: 2023-06-22T18:12:13.407+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:13.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:13 smithi101 conmon[143689]: 2023-06-22T18:12:13.407+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:13.930 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 systemd[1]: Stopping Ceph osd.8 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:13.931 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.613+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:13.931 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.613+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:13.931 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.613+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:13.931 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.613+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:13.931 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.613+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:13.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:13 smithi039 conmon[147329]: 2023-06-22T18:12:13.648+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:13.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:13 smithi039 conmon[147329]: 2023-06-22T18:12:13.648+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:13.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:13 smithi039 conmon[147329]: 2023-06-22T18:12:13.648+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:13.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:13 smithi039 conmon[147329]: 2023-06-22T18:12:13.648+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:13.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:13 smithi039 conmon[147329]: 2023-06-22T18:12:13.648+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:14.234 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.927+0000 7f6234e6e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:14.234 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.927+0000 7f6234e6e700 -1 osd.8 85 *** Got signal Terminated *** 2023-06-22T18:12:14.234 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:13 smithi133 conmon[138513]: 2023-06-22T18:12:13.927+0000 7f6234e6e700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:12:14.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[145608]: 2023-06-22T18:12:14.144+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:14.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[145608]: 2023-06-22T18:12:14.144+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:14.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[145608]: 2023-06-22T18:12:14.144+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:14.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[145608]: 2023-06-22T18:12:14.144+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:14.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:14 smithi101 conmon[143689]: 2023-06-22T18:12:14.414+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:14.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:14 smithi101 conmon[143689]: 2023-06-22T18:12:14.415+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:14.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:14 smithi101 conmon[143689]: 2023-06-22T18:12:14.415+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:14.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:14 smithi101 conmon[143689]: 2023-06-22T18:12:14.415+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:14.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:14 smithi101 conmon[143689]: 2023-06-22T18:12:14.415+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:14.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:14 smithi039 conmon[147329]: 2023-06-22T18:12:14.605+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:14.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:14 smithi039 conmon[147329]: 2023-06-22T18:12:14.605+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:14.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:14 smithi039 conmon[147329]: 2023-06-22T18:12:14.605+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:14.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:14 smithi039 conmon[147329]: 2023-06-22T18:12:14.605+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:14.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:14 smithi039 conmon[147329]: 2023-06-22T18:12:14.605+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:14.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[138513]: 2023-06-22T18:12:14.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:14.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[138513]: 2023-06-22T18:12:14.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:14.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[138513]: 2023-06-22T18:12:14.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:14.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[138513]: 2023-06-22T18:12:14.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:14.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:14 smithi133 conmon[138513]: 2023-06-22T18:12:14.655+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:15.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[145608]: 2023-06-22T18:12:15.116+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:15.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[145608]: 2023-06-22T18:12:15.116+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:15.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[145608]: 2023-06-22T18:12:15.116+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:15.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[145608]: 2023-06-22T18:12:15.116+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:15.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:15 smithi101 conmon[143689]: 2023-06-22T18:12:15.464+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:15.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:15 smithi101 conmon[143689]: 2023-06-22T18:12:15.464+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:15.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:15 smithi101 conmon[143689]: 2023-06-22T18:12:15.464+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:15.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:15 smithi101 conmon[143689]: 2023-06-22T18:12:15.464+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:15.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:15 smithi101 conmon[143689]: 2023-06-22T18:12:15.464+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:15.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:15 smithi039 conmon[147329]: 2023-06-22T18:12:15.652+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:15.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:15 smithi039 conmon[147329]: 2023-06-22T18:12:15.652+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:15.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:15 smithi039 conmon[147329]: 2023-06-22T18:12:15.652+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:15.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:15 smithi039 conmon[147329]: 2023-06-22T18:12:15.652+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:15.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:15 smithi039 conmon[147329]: 2023-06-22T18:12:15.652+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:15.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[138513]: 2023-06-22T18:12:15.689+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:15.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[138513]: 2023-06-22T18:12:15.689+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:15.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[138513]: 2023-06-22T18:12:15.689+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:15.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[138513]: 2023-06-22T18:12:15.689+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:15.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:15 smithi133 conmon[138513]: 2023-06-22T18:12:15.689+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:16.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[145608]: 2023-06-22T18:12:16.073+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:16.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[145608]: 2023-06-22T18:12:16.073+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:16.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[145608]: 2023-06-22T18:12:16.074+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:16.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[145608]: 2023-06-22T18:12:16.074+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:16.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:16 smithi101 conmon[143689]: 2023-06-22T18:12:16.422+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:16.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:16 smithi101 conmon[143689]: 2023-06-22T18:12:16.422+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:16.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:16 smithi101 conmon[143689]: 2023-06-22T18:12:16.422+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:16.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:16 smithi101 conmon[143689]: 2023-06-22T18:12:16.422+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:16.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:16 smithi101 conmon[143689]: 2023-06-22T18:12:16.422+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:16.854 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:16 smithi101 conmon[143689]: 2023-06-22T18:12:16.422+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:16.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:16 smithi039 conmon[147329]: 2023-06-22T18:12:16.641+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:16.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:16 smithi039 conmon[147329]: 2023-06-22T18:12:16.641+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:16.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:16 smithi039 conmon[147329]: 2023-06-22T18:12:16.641+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:16.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:16 smithi039 conmon[147329]: 2023-06-22T18:12:16.641+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:16.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:16 smithi039 conmon[147329]: 2023-06-22T18:12:16.641+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:16.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:16 smithi039 conmon[147329]: 2023-06-22T18:12:16.641+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:16.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[138513]: 2023-06-22T18:12:16.730+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:16.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[138513]: 2023-06-22T18:12:16.730+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:16.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[138513]: 2023-06-22T18:12:16.730+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:16.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[138513]: 2023-06-22T18:12:16.730+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:16.985 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:16 smithi133 conmon[138513]: 2023-06-22T18:12:16.730+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:17.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[145608]: 2023-06-22T18:12:17.066+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:17.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[145608]: 2023-06-22T18:12:17.066+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:17.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[145608]: 2023-06-22T18:12:17.066+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:17.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[145608]: 2023-06-22T18:12:17.066+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:17.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:17 smithi101 conmon[143689]: 2023-06-22T18:12:17.411+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:17.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:17 smithi101 conmon[143689]: 2023-06-22T18:12:17.411+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:17.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:17 smithi101 conmon[143689]: 2023-06-22T18:12:17.411+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:17.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:17 smithi101 conmon[143689]: 2023-06-22T18:12:17.411+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:17.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:17 smithi101 conmon[143689]: 2023-06-22T18:12:17.411+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:17.854 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:17 smithi101 conmon[143689]: 2023-06-22T18:12:17.411+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:17.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:17 smithi039 conmon[147329]: 2023-06-22T18:12:17.645+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:17.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:17 smithi039 conmon[147329]: 2023-06-22T18:12:17.645+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:17.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:17 smithi039 conmon[147329]: 2023-06-22T18:12:17.645+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:17.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:17 smithi039 conmon[147329]: 2023-06-22T18:12:17.645+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:17.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:17 smithi039 conmon[147329]: 2023-06-22T18:12:17.645+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:17.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:17 smithi039 conmon[147329]: 2023-06-22T18:12:17.645+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:17.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[138513]: 2023-06-22T18:12:17.707+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:17.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[138513]: 2023-06-22T18:12:17.707+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:17.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[138513]: 2023-06-22T18:12:17.707+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:17.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[138513]: 2023-06-22T18:12:17.707+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:17.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:17 smithi133 conmon[138513]: 2023-06-22T18:12:17.707+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:18.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[145608]: 2023-06-22T18:12:18.023+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:18.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[145608]: 2023-06-22T18:12:18.023+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:18.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[145608]: 2023-06-22T18:12:18.023+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:18.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[145608]: 2023-06-22T18:12:18.023+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:18.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:18 smithi101 conmon[143689]: 2023-06-22T18:12:18.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:18.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:18 smithi101 conmon[143689]: 2023-06-22T18:12:18.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:18.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:18 smithi101 conmon[143689]: 2023-06-22T18:12:18.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:18.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:18 smithi101 conmon[143689]: 2023-06-22T18:12:18.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:18.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:18 smithi101 conmon[143689]: 2023-06-22T18:12:18.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:18.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:18 smithi101 conmon[143689]: 2023-06-22T18:12:18.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:18.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:18 smithi039 conmon[147329]: 2023-06-22T18:12:18.650+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:18.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:18 smithi039 conmon[147329]: 2023-06-22T18:12:18.650+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:18.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:18 smithi039 conmon[147329]: 2023-06-22T18:12:18.650+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:18.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:18 smithi039 conmon[147329]: 2023-06-22T18:12:18.650+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:18.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:18 smithi039 conmon[147329]: 2023-06-22T18:12:18.650+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:18.945 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:18 smithi039 conmon[147329]: 2023-06-22T18:12:18.650+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:18.983 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[138513]: 2023-06-22T18:12:18.729+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.812630+0000 front 2023-06-22T18:11:11.812358+0000 (oldest deadline 2023-06-22T18:11:37.712518+0000) 2023-06-22T18:12:18.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[138513]: 2023-06-22T18:12:18.729+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:22.413571+0000 front 2023-06-22T18:11:22.413607+0000 (oldest deadline 2023-06-22T18:11:47.713666+0000) 2023-06-22T18:12:18.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[138513]: 2023-06-22T18:12:18.729+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:27.714114+0000 front 2023-06-22T18:11:27.714134+0000 (oldest deadline 2023-06-22T18:11:53.014305+0000) 2023-06-22T18:12:18.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[138513]: 2023-06-22T18:12:18.729+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:38.915325+0000 front 2023-06-22T18:11:38.915473+0000 (oldest deadline 2023-06-22T18:12:04.815406+0000) 2023-06-22T18:12:18.984 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:18 smithi133 conmon[138513]: 2023-06-22T18:12:18.729+0000 7f622e466700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.816124+0000 front 2023-06-22T18:11:44.816035+0000 (oldest deadline 2023-06-22T18:12:08.915905+0000) 2023-06-22T18:12:19.479 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:19.479 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:19.480 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:19.480 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:19.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:19 smithi101 conmon[143689]: 2023-06-22T18:12:19.447+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:19.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:19 smithi101 conmon[143689]: 2023-06-22T18:12:19.447+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:19.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:19 smithi101 conmon[143689]: 2023-06-22T18:12:19.447+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:19.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:19 smithi101 conmon[143689]: 2023-06-22T18:12:19.447+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:19.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:19 smithi101 conmon[143689]: 2023-06-22T18:12:19.447+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:19.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:19 smithi101 conmon[143689]: 2023-06-22T18:12:19.447+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:20.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:19 smithi039 conmon[147329]: 2023-06-22T18:12:19.698+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:20.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:19 smithi039 conmon[147329]: 2023-06-22T18:12:19.698+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:20.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:19 smithi039 conmon[147329]: 2023-06-22T18:12:19.698+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:20.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:19 smithi039 conmon[147329]: 2023-06-22T18:12:19.698+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:20.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:19 smithi039 conmon[147329]: 2023-06-22T18:12:19.698+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:20.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:19 smithi039 conmon[147329]: 2023-06-22T18:12:19.698+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:20.233 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jun 22 18:12:19 smithi133 bash[155126]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-8 2023-06-22T18:12:20.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.994+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:20.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.994+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:20.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.994+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:20.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.994+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:20.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:19 smithi133 conmon[145608]: 2023-06-22T18:12:19.994+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:20.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:20 smithi101 conmon[143689]: 2023-06-22T18:12:20.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:20.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:20 smithi101 conmon[143689]: 2023-06-22T18:12:20.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:20.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:20 smithi101 conmon[143689]: 2023-06-22T18:12:20.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:20.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:20 smithi101 conmon[143689]: 2023-06-22T18:12:20.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:20.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:20 smithi101 conmon[143689]: 2023-06-22T18:12:20.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:20.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:20 smithi101 conmon[143689]: 2023-06-22T18:12:20.400+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:21.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:20 smithi039 conmon[147329]: 2023-06-22T18:12:20.700+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:21.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:20 smithi039 conmon[147329]: 2023-06-22T18:12:20.700+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:21.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:20 smithi039 conmon[147329]: 2023-06-22T18:12:20.700+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:21.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:20 smithi039 conmon[147329]: 2023-06-22T18:12:20.700+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:21.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:20 smithi039 conmon[147329]: 2023-06-22T18:12:20.701+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:21.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:20 smithi039 conmon[147329]: 2023-06-22T18:12:20.701+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:21.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:20 smithi133 conmon[145608]: 2023-06-22T18:12:20.988+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:21.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:20 smithi133 conmon[145608]: 2023-06-22T18:12:20.988+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:21.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:20 smithi133 conmon[145608]: 2023-06-22T18:12:20.988+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:21.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:20 smithi133 conmon[145608]: 2023-06-22T18:12:20.988+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:21.326 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:20 smithi133 conmon[145608]: 2023-06-22T18:12:20.988+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:21.332 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.8.service' 2023-06-22T18:12:21.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:21.370 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-06-22T18:12:21.371 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-06-22T18:12:21.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.9 2023-06-22T18:12:21.694 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 systemd[1]: Stopping Ceph osd.9 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:21.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:21 smithi101 conmon[143689]: 2023-06-22T18:12:21.408+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:21.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:21 smithi101 conmon[143689]: 2023-06-22T18:12:21.408+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:21.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:21 smithi101 conmon[143689]: 2023-06-22T18:12:21.408+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:21.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:21 smithi101 conmon[143689]: 2023-06-22T18:12:21.408+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:21.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:21 smithi101 conmon[143689]: 2023-06-22T18:12:21.408+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:21.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:21 smithi101 conmon[143689]: 2023-06-22T18:12:21.408+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:22.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.740+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:22.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.740+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.740+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.740+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.740+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.740+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.890+0000 7fbc8d4a1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.890+0000 7fbc8d4a1700 -1 osd.9 85 *** Got signal Terminated *** 2023-06-22T18:12:22.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:21 smithi039 conmon[147329]: 2023-06-22T18:12:21.890+0000 7fbc8d4a1700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:12:22.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:21 smithi133 conmon[145608]: 2023-06-22T18:12:21.983+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:22.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:21 smithi133 conmon[145608]: 2023-06-22T18:12:21.983+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:22.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:21 smithi133 conmon[145608]: 2023-06-22T18:12:21.983+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:22.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:21 smithi133 conmon[145608]: 2023-06-22T18:12:21.983+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:22.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:21 smithi133 conmon[145608]: 2023-06-22T18:12:21.983+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:22.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:22 smithi101 conmon[143689]: 2023-06-22T18:12:22.397+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:22.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:22 smithi101 conmon[143689]: 2023-06-22T18:12:22.397+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:22.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:22 smithi101 conmon[143689]: 2023-06-22T18:12:22.397+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:22.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:22 smithi101 conmon[143689]: 2023-06-22T18:12:22.397+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:22.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:22 smithi101 conmon[143689]: 2023-06-22T18:12:22.397+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:22.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:22 smithi101 conmon[143689]: 2023-06-22T18:12:22.397+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:23.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:22 smithi039 conmon[147329]: 2023-06-22T18:12:22.754+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:23.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:22 smithi039 conmon[147329]: 2023-06-22T18:12:22.754+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:23.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:22 smithi039 conmon[147329]: 2023-06-22T18:12:22.754+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:23.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:22 smithi039 conmon[147329]: 2023-06-22T18:12:22.754+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:23.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:22 smithi039 conmon[147329]: 2023-06-22T18:12:22.754+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:23.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:22 smithi039 conmon[147329]: 2023-06-22T18:12:22.754+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:23.233 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:22 smithi133 conmon[145608]: 2023-06-22T18:12:22.946+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:23.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:22 smithi133 conmon[145608]: 2023-06-22T18:12:22.946+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:23.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:22 smithi133 conmon[145608]: 2023-06-22T18:12:22.946+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:23.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:22 smithi133 conmon[145608]: 2023-06-22T18:12:22.946+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:23.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:22 smithi133 conmon[145608]: 2023-06-22T18:12:22.946+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:23.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:23 smithi101 conmon[143689]: 2023-06-22T18:12:23.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:23.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:23 smithi101 conmon[143689]: 2023-06-22T18:12:23.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:23.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:23 smithi101 conmon[143689]: 2023-06-22T18:12:23.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:23.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:23 smithi101 conmon[143689]: 2023-06-22T18:12:23.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:23.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:23 smithi101 conmon[143689]: 2023-06-22T18:12:23.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:23.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:23 smithi101 conmon[143689]: 2023-06-22T18:12:23.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:24.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:23 smithi039 conmon[147329]: 2023-06-22T18:12:23.793+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:24.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:23 smithi039 conmon[147329]: 2023-06-22T18:12:23.793+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:24.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:23 smithi039 conmon[147329]: 2023-06-22T18:12:23.793+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:24.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:23 smithi039 conmon[147329]: 2023-06-22T18:12:23.793+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:24.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:23 smithi039 conmon[147329]: 2023-06-22T18:12:23.793+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:24.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:23 smithi039 conmon[147329]: 2023-06-22T18:12:23.793+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:24.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:23 smithi133 conmon[145608]: 2023-06-22T18:12:23.979+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:24.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:23 smithi133 conmon[145608]: 2023-06-22T18:12:23.979+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:24.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:23 smithi133 conmon[145608]: 2023-06-22T18:12:23.979+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:24.234 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:23 smithi133 conmon[145608]: 2023-06-22T18:12:23.979+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:24.235 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:23 smithi133 conmon[145608]: 2023-06-22T18:12:23.979+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:24.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:24.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:24.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:24.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:24.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:24.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:24.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:24 smithi101 conmon[143689]: 2023-06-22T18:12:24.406+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:25.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:25.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:25.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:25.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:25.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:25.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:25.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:24 smithi039 conmon[147329]: 2023-06-22T18:12:24.821+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:58.725971+0000 front 2023-06-22T18:11:58.726059+0000 (oldest deadline 2023-06-22T18:12:24.626125+0000) 2023-06-22T18:12:25.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:24 smithi133 conmon[145608]: 2023-06-22T18:12:24.993+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:25.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:24 smithi133 conmon[145608]: 2023-06-22T18:12:24.993+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:25.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:24 smithi133 conmon[145608]: 2023-06-22T18:12:24.993+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:25.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:24 smithi133 conmon[145608]: 2023-06-22T18:12:24.993+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:25.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:24 smithi133 conmon[145608]: 2023-06-22T18:12:24.993+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:25.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:25.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:25.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:25.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:25.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:25.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:25.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:25 smithi101 conmon[143689]: 2023-06-22T18:12:25.441+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:26.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:26.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:26.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:26.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:26.196 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:26.196 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:26.196 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:25 smithi039 conmon[147329]: 2023-06-22T18:12:25.828+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:58.725971+0000 front 2023-06-22T18:11:58.726059+0000 (oldest deadline 2023-06-22T18:12:24.626125+0000) 2023-06-22T18:12:26.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:26 smithi133 conmon[145608]: 2023-06-22T18:12:26.008+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:26.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:26 smithi133 conmon[145608]: 2023-06-22T18:12:26.008+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:26.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:26 smithi133 conmon[145608]: 2023-06-22T18:12:26.008+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:26.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:26 smithi133 conmon[145608]: 2023-06-22T18:12:26.008+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:26.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:26 smithi133 conmon[145608]: 2023-06-22T18:12:26.008+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:26.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:26.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:26.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:26.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:26.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:26.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:26.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:26 smithi101 conmon[143689]: 2023-06-22T18:12:26.412+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:27.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:11.815594+0000 front 2023-06-22T18:11:11.815643+0000 (oldest deadline 2023-06-22T18:11:40.616417+0000) 2023-06-22T18:12:27.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:20.617299+0000 front 2023-06-22T18:11:20.617170+0000 (oldest deadline 2023-06-22T18:11:46.517176+0000) 2023-06-22T18:12:27.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.218782+0000 front 2023-06-22T18:11:31.218767+0000 (oldest deadline 2023-06-22T18:11:54.718727+0000) 2023-06-22T18:12:27.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:36.920403+0000 front 2023-06-22T18:11:36.920538+0000 (oldest deadline 2023-06-22T18:11:59.820524+0000) 2023-06-22T18:12:27.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:45.522716+0000 front 2023-06-22T18:11:45.522947+0000 (oldest deadline 2023-06-22T18:12:08.422751+0000) 2023-06-22T18:12:27.194 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.125173+0000 front 2023-06-22T18:11:54.125222+0000 (oldest deadline 2023-06-22T18:12:15.825116+0000) 2023-06-22T18:12:27.195 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:26 smithi039 conmon[147329]: 2023-06-22T18:12:26.876+0000 7fbc88a9d700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:58.725971+0000 front 2023-06-22T18:11:58.726059+0000 (oldest deadline 2023-06-22T18:12:24.626125+0000) 2023-06-22T18:12:27.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:27 smithi133 conmon[145608]: 2023-06-22T18:12:27.049+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:27.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:27 smithi133 conmon[145608]: 2023-06-22T18:12:27.049+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:27.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:27 smithi133 conmon[145608]: 2023-06-22T18:12:27.049+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:27.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:27 smithi133 conmon[145608]: 2023-06-22T18:12:27.049+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:27.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:27 smithi133 conmon[145608]: 2023-06-22T18:12:27.049+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:27.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:27 smithi133 conmon[145608]: 2023-06-22T18:12:27.049+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:27.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:27.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:27.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:27.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:27.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:27.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:27.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:27 smithi101 conmon[143689]: 2023-06-22T18:12:27.369+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:27.944 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jun 22 18:12:27 smithi039 bash[159536]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-9 2023-06-22T18:12:28.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:28 smithi133 conmon[145608]: 2023-06-22T18:12:28.078+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:28.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:28 smithi133 conmon[145608]: 2023-06-22T18:12:28.078+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:28.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:28 smithi133 conmon[145608]: 2023-06-22T18:12:28.078+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:28.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:28 smithi133 conmon[145608]: 2023-06-22T18:12:28.078+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:28.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:28 smithi133 conmon[145608]: 2023-06-22T18:12:28.078+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:28.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:28 smithi133 conmon[145608]: 2023-06-22T18:12:28.078+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:28.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:28.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:28.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:28.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:28.854 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:28.854 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:28.854 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:28 smithi101 conmon[143689]: 2023-06-22T18:12:28.360+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:29.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:29 smithi133 conmon[145608]: 2023-06-22T18:12:29.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:29.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:29 smithi133 conmon[145608]: 2023-06-22T18:12:29.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:29.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:29 smithi133 conmon[145608]: 2023-06-22T18:12:29.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:29.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:29 smithi133 conmon[145608]: 2023-06-22T18:12:29.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:29.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:29 smithi133 conmon[145608]: 2023-06-22T18:12:29.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:29.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:29 smithi133 conmon[145608]: 2023-06-22T18:12:29.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:29.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.9.service' 2023-06-22T18:12:29.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:29.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:29.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:29.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:29.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:29.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:29.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.316+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:29.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:29.639 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-06-22T18:12:29.639 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-06-22T18:12:29.639 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.10 2023-06-22T18:12:29.979 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 systemd[1]: Stopping Ceph osd.10 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:30.352 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.976+0000 7fca1cbf0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:30.353 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.976+0000 7fca1cbf0700 -1 osd.10 85 *** Got signal Terminated *** 2023-06-22T18:12:30.353 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:29 smithi101 conmon[143689]: 2023-06-22T18:12:29.976+0000 7fca1cbf0700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:12:30.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:30 smithi133 conmon[145608]: 2023-06-22T18:12:30.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:30.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:30 smithi133 conmon[145608]: 2023-06-22T18:12:30.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:30.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:30 smithi133 conmon[145608]: 2023-06-22T18:12:30.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:30.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:30 smithi133 conmon[145608]: 2023-06-22T18:12:30.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:30.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:30 smithi133 conmon[145608]: 2023-06-22T18:12:30.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:30.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:30 smithi133 conmon[145608]: 2023-06-22T18:12:30.076+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:30.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:30.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:30.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:30.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:30.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:30.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:30.853 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:30 smithi101 conmon[143689]: 2023-06-22T18:12:30.350+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:31.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:31 smithi133 conmon[145608]: 2023-06-22T18:12:31.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:31.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:31 smithi133 conmon[145608]: 2023-06-22T18:12:31.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:31.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:31 smithi133 conmon[145608]: 2023-06-22T18:12:31.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:31.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:31 smithi133 conmon[145608]: 2023-06-22T18:12:31.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:31.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:31 smithi133 conmon[145608]: 2023-06-22T18:12:31.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:31.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:31 smithi133 conmon[145608]: 2023-06-22T18:12:31.027+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:31.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:31.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:31.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:31.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:31.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:31.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:31.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:31 smithi101 conmon[143689]: 2023-06-22T18:12:31.338+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:32.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:32 smithi133 conmon[145608]: 2023-06-22T18:12:32.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:32.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:32 smithi133 conmon[145608]: 2023-06-22T18:12:32.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:32.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:32 smithi133 conmon[145608]: 2023-06-22T18:12:32.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:32.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:32 smithi133 conmon[145608]: 2023-06-22T18:12:32.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:32.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:32 smithi133 conmon[145608]: 2023-06-22T18:12:32.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:32.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:32 smithi133 conmon[145608]: 2023-06-22T18:12:32.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:32.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:32.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:32.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:32.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:32.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:32.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:32.604 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:32 smithi101 conmon[143689]: 2023-06-22T18:12:32.307+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:33.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:33 smithi133 conmon[145608]: 2023-06-22T18:12:33.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:33.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:33 smithi133 conmon[145608]: 2023-06-22T18:12:33.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:33.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:33 smithi133 conmon[145608]: 2023-06-22T18:12:33.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:33.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:33 smithi133 conmon[145608]: 2023-06-22T18:12:33.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:33.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:33 smithi133 conmon[145608]: 2023-06-22T18:12:33.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:33.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:33 smithi133 conmon[145608]: 2023-06-22T18:12:33.032+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:33.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:33.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:33.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:33.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:33.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:33.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:33.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:33 smithi101 conmon[143689]: 2023-06-22T18:12:33.265+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:34.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:34 smithi133 conmon[145608]: 2023-06-22T18:12:34.081+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:34.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:34 smithi133 conmon[145608]: 2023-06-22T18:12:34.081+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:34.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:34 smithi133 conmon[145608]: 2023-06-22T18:12:34.081+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:34.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:34 smithi133 conmon[145608]: 2023-06-22T18:12:34.081+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:34.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:34 smithi133 conmon[145608]: 2023-06-22T18:12:34.081+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:34.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:34 smithi133 conmon[145608]: 2023-06-22T18:12:34.081+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:34.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:15.043490+0000 front 2023-06-22T18:11:15.043460+0000 (oldest deadline 2023-06-22T18:11:38.543302+0000) 2023-06-22T18:12:34.602 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.444288+0000 front 2023-06-22T18:11:21.444264+0000 (oldest deadline 2023-06-22T18:11:47.344464+0000) 2023-06-22T18:12:34.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:29.045582+0000 front 2023-06-22T18:11:29.045662+0000 (oldest deadline 2023-06-22T18:11:53.745563+0000) 2023-06-22T18:12:34.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2023-06-22T18:11:37.747362+0000 front 2023-06-22T18:11:37.747487+0000 (oldest deadline 2023-06-22T18:12:00.647292+0000) 2023-06-22T18:12:34.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:47.048651+0000 front 2023-06-22T18:11:47.048667+0000 (oldest deadline 2023-06-22T18:12:09.948714+0000) 2023-06-22T18:12:34.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:54.550322+0000 front 2023-06-22T18:11:54.550425+0000 (oldest deadline 2023-06-22T18:12:16.250242+0000) 2023-06-22T18:12:34.603 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:34 smithi101 conmon[143689]: 2023-06-22T18:12:34.235+0000 7fca1a1f0700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:11:59.751046+0000 front 2023-06-22T18:11:59.751084+0000 (oldest deadline 2023-06-22T18:12:23.851052+0000) 2023-06-22T18:12:35.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:35 smithi133 conmon[145608]: 2023-06-22T18:12:35.119+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:35.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:35 smithi133 conmon[145608]: 2023-06-22T18:12:35.119+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:35.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:35 smithi133 conmon[145608]: 2023-06-22T18:12:35.119+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:35.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:35 smithi133 conmon[145608]: 2023-06-22T18:12:35.119+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:35.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:35 smithi133 conmon[145608]: 2023-06-22T18:12:35.119+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:35.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:35 smithi133 conmon[145608]: 2023-06-22T18:12:35.119+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:35.852 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jun 22 18:12:35 smithi101 bash[153458]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-10 2023-06-22T18:12:36.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:36.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:36.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:36.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:36.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:36.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:36.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:36 smithi133 conmon[145608]: 2023-06-22T18:12:36.160+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:37.391 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.10.service' 2023-06-22T18:12:37.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:37.429 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-06-22T18:12:37.429 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-06-22T18:12:37.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.11 2023-06-22T18:12:37.457 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:37.457 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:37.458 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:37.458 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:37.458 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:37.458 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:37.458 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.125+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:37.733 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 systemd[1]: Stopping Ceph osd.11 for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:38.148 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.782+0000 7efca0690700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:38.148 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.782+0000 7efca0690700 -1 osd.11 85 *** Got signal Terminated *** 2023-06-22T18:12:38.148 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:37 smithi133 conmon[145608]: 2023-06-22T18:12:37.782+0000 7efca0690700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-22T18:12:38.148 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:38.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:38.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:38.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:38.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:38.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:38.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:38 smithi133 conmon[145608]: 2023-06-22T18:12:38.145+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:39.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.106+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:39.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:39.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:39.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:39.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:39.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:39.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:39 smithi133 conmon[145608]: 2023-06-22T18:12:39.107+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:40.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:40.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:40.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:40.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:40.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:40.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:40.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:40 smithi133 conmon[145608]: 2023-06-22T18:12:40.099+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:41.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:41.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:41.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:41.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:41.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:41.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:41.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:41 smithi133 conmon[145608]: 2023-06-22T18:12:41.113+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:42.483 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2023-06-22T18:11:13.425935+0000 front 2023-06-22T18:11:13.426075+0000 (oldest deadline 2023-06-22T18:11:36.925808+0000) 2023-06-22T18:12:42.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-06-22T18:11:21.026953+0000 front 2023-06-22T18:11:21.027047+0000 (oldest deadline 2023-06-22T18:11:45.726925+0000) 2023-06-22T18:12:42.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2023-06-22T18:11:31.027765+0000 front 2023-06-22T18:11:31.027820+0000 (oldest deadline 2023-06-22T18:11:55.727936+0000) 2023-06-22T18:12:42.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-06-22T18:11:44.529625+0000 front 2023-06-22T18:11:44.529570+0000 (oldest deadline 2023-06-22T18:12:08.629529+0000) 2023-06-22T18:12:42.484 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2023-06-22T18:11:53.831128+0000 front 2023-06-22T18:11:53.831265+0000 (oldest deadline 2023-06-22T18:12:19.731303+0000) 2023-06-22T18:12:42.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2023-06-22T18:12:01.332762+0000 front 2023-06-22T18:12:01.332780+0000 (oldest deadline 2023-06-22T18:12:26.632610+0000) 2023-06-22T18:12:42.485 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:42 smithi133 conmon[145608]: 2023-06-22T18:12:42.143+0000 7efc99c88700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-06-22T18:12:10.733835+0000 front 2023-06-22T18:12:10.733725+0000 (oldest deadline 2023-06-22T18:12:36.033624+0000) 2023-06-22T18:12:43.983 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jun 22 18:12:43 smithi133 bash[155675]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-osd-11 2023-06-22T18:12:45.078 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@osd.11.service' 2023-06-22T18:12:45.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:45.157 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-06-22T18:12:45.158 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-06-22T18:12:45.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.a 2023-06-22T18:12:45.673 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:12:45 smithi039 systemd[1]: Stopping Ceph mds.a for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:45.944 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:12:45 smithi039 conmon[154468]: 2023-06-22T18:12:45.670+0000 7f7e5f1f5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:45.944 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:12:45 smithi039 conmon[154468]: 2023-06-22T18:12:45.670+0000 7f7e5f1f5700 -1 mds.a *** got signal Terminated *** 2023-06-22T18:12:50.194 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jun 22 18:12:50 smithi039 bash[160086]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-a 2023-06-22T18:12:50.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.a.service' 2023-06-22T18:12:50.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:50.392 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-06-22T18:12:50.393 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-06-22T18:12:50.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.d 2023-06-22T18:12:50.694 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:12:50 smithi039 systemd[1]: Stopping Ceph mds.d for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:51.194 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:12:50 smithi039 conmon[152990]: 2023-06-22T18:12:50.902+0000 7fa96251f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:51.194 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:12:50 smithi039 conmon[152990]: 2023-06-22T18:12:50.902+0000 7fa96251f700 -1 mds.d *** got signal Terminated *** 2023-06-22T18:12:55.444 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jun 22 18:12:55 smithi039 bash[160372]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-d 2023-06-22T18:12:55.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.d.service' 2023-06-22T18:12:55.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:12:55.692 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-06-22T18:12:55.693 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-06-22T18:12:55.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.g 2023-06-22T18:12:55.944 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:12:55 smithi039 systemd[1]: Stopping Ceph mds.g for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:12:56.444 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:12:56 smithi039 conmon[153857]: 2023-06-22T18:12:56.184+0000 7f5901fc2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:12:56.444 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:12:56 smithi039 conmon[153857]: 2023-06-22T18:12:56.184+0000 7f5901fc2700 -1 mds.g *** got signal Terminated *** 2023-06-22T18:12:59.944 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jun 22 18:12:59 smithi039 bash[160654]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-g 2023-06-22T18:13:00.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.g.service' 2023-06-22T18:13:00.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:00.102 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-06-22T18:13:00.102 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-06-22T18:13:00.103 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.j 2023-06-22T18:13:00.443 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:13:00 smithi039 systemd[1]: Stopping Ceph mds.j for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:00.944 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:13:00 smithi039 conmon[152380]: 2023-06-22T18:13:00.591+0000 7f9af7820700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:00.944 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:13:00 smithi039 conmon[152380]: 2023-06-22T18:13:00.591+0000 7f9af7820700 -1 mds.j *** got signal Terminated *** 2023-06-22T18:13:04.694 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jun 22 18:13:04 smithi039 bash[160938]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-j 2023-06-22T18:13:04.792 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.j.service' 2023-06-22T18:13:04.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:04.870 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-06-22T18:13:04.871 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-06-22T18:13:04.872 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.b 2023-06-22T18:13:05.215 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:13:04 smithi101 systemd[1]: Stopping Ceph mds.b for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:05.602 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:13:05 smithi101 conmon[148913]: 2023-06-22T18:13:05.212+0000 7fc4bfbec700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:05.602 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:13:05 smithi101 conmon[148913]: 2023-06-22T18:13:05.212+0000 7fc4bfbec700 -1 mds.b *** got signal Terminated *** 2023-06-22T18:13:10.352 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jun 22 18:13:10 smithi101 bash[154003]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-b 2023-06-22T18:13:10.434 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.b.service' 2023-06-22T18:13:10.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:10.469 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-06-22T18:13:10.469 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-06-22T18:13:10.469 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.e 2023-06-22T18:13:10.827 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:13:10 smithi101 systemd[1]: Stopping Ceph mds.e for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:11.102 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:13:10 smithi101 conmon[147782]: 2023-06-22T18:13:10.824+0000 7f277f78e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:11.102 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:13:10 smithi101 conmon[147782]: 2023-06-22T18:13:10.824+0000 7f277f78e700 -1 mds.e *** got signal Terminated *** 2023-06-22T18:13:14.602 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jun 22 18:13:14 smithi101 bash[154291]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-e 2023-06-22T18:13:14.766 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.e.service' 2023-06-22T18:13:14.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:14.797 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-06-22T18:13:14.798 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-06-22T18:13:14.798 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.h 2023-06-22T18:13:15.102 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:13:14 smithi101 systemd[1]: Stopping Ceph mds.h for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:15.602 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:13:15 smithi101 conmon[148348]: 2023-06-22T18:13:15.148+0000 7fefde0c4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:15.602 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:13:15 smithi101 conmon[148348]: 2023-06-22T18:13:15.148+0000 7fefde0c4700 -1 mds.h *** got signal Terminated *** 2023-06-22T18:13:18.102 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jun 22 18:13:17 smithi101 bash[154576]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-h 2023-06-22T18:13:18.219 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.h.service' 2023-06-22T18:13:18.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:18.252 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-06-22T18:13:18.253 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-06-22T18:13:18.253 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.k 2023-06-22T18:13:18.602 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:13:18 smithi101 systemd[1]: Stopping Ceph mds.k for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:19.102 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:13:18 smithi101 conmon[147217]: 2023-06-22T18:13:18.606+0000 7f85e5f8a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:19.102 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:13:18 smithi101 conmon[147217]: 2023-06-22T18:13:18.606+0000 7f85e5f8a700 -1 mds.k *** got signal Terminated *** 2023-06-22T18:13:21.745 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jun 22 18:13:21 smithi101 bash[154874]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-k 2023-06-22T18:13:21.812 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.k.service' 2023-06-22T18:13:21.843 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:21.843 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-06-22T18:13:21.844 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-06-22T18:13:21.844 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.c 2023-06-22T18:13:22.233 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:13:21 smithi133 systemd[1]: Stopping Ceph mds.c for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:22.733 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:13:22 smithi133 conmon[150594]: 2023-06-22T18:13:22.357+0000 7f8cb01fa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:22.734 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:13:22 smithi133 conmon[150594]: 2023-06-22T18:13:22.357+0000 7f8cb01fa700 -1 mds.c *** got signal Terminated *** 2023-06-22T18:13:26.734 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jun 22 18:13:26 smithi133 bash[156228]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-c 2023-06-22T18:13:26.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.c.service' 2023-06-22T18:13:26.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:26.936 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-06-22T18:13:26.936 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-06-22T18:13:26.936 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.f 2023-06-22T18:13:27.233 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:13:27 smithi133 systemd[1]: Stopping Ceph mds.f for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:27.734 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:13:27 smithi133 conmon[150033]: 2023-06-22T18:13:27.298+0000 7f1aff238700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:27.734 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jun 22 18:13:27 smithi133 conmon[150033]: 2023-06-22T18:13:27.298+0000 7f1aff238700 -1 mds.f *** got signal Terminated *** 2023-06-22T18:13:29.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.f.service' 2023-06-22T18:13:29.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:29.239 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-06-22T18:13:29.240 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-06-22T18:13:29.240 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.i 2023-06-22T18:13:29.616 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:13:29 smithi133 systemd[1]: Stopping Ceph mds.i for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:29.983 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:13:29 smithi133 conmon[151159]: 2023-06-22T18:13:29.613+0000 7f4259af4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:29.983 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:13:29 smithi133 conmon[151159]: 2023-06-22T18:13:29.613+0000 7f4259af4700 -1 mds.i *** got signal Terminated *** 2023-06-22T18:13:32.483 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jun 22 18:13:32 smithi133 bash[156780]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-i 2023-06-22T18:13:32.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.i.service' 2023-06-22T18:13:32.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:32.710 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-06-22T18:13:32.711 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-06-22T18:13:32.711 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.l 2023-06-22T18:13:32.983 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:13:32 smithi133 systemd[1]: Stopping Ceph mds.l for d572fea0-1126-11ee-9b2b-001a4aab830c... 2023-06-22T18:13:33.483 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:13:33 smithi133 conmon[149468]: 2023-06-22T18:13:33.059+0000 7fdda3091700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-22T18:13:33.484 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:13:33 smithi133 conmon[149468]: 2023-06-22T18:13:33.060+0000 7fdda3091700 -1 mds.l *** got signal Terminated *** 2023-06-22T18:13:35.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.l.service' 2023-06-22T18:13:35.225 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:13:34 smithi133 bash[157063]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c-mds-l 2023-06-22T18:13:35.226 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:13:35 smithi133 systemd[1]: ceph-d572fea0-1126-11ee-9b2b-001a4aab830c@mds.l.service: Succeeded. 2023-06-22T18:13:35.226 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jun 22 18:13:35 smithi133 systemd[1]: Stopped Ceph mds.l for d572fea0-1126-11ee-9b2b-001a4aab830c. 2023-06-22T18:13:35.234 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-22T18:13:35.234 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-06-22T18:13:35.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-22T18:13:39.143 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-22T18:13:41.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-22T18:13:42.997 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:13:43.069 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:13:43.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-22T18:13:43.128 INFO:tasks.cephadm:Archiving crash dumps... 2023-06-22T18:13:43.130 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi039/crash 2023-06-22T18:13:43.132 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash -- . 2023-06-22T18:13:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-22T18:13:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2023-06-22T18:13:43.305 DEBUG:teuthology.misc:Transferring archived files from smithi101:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi101/crash 2023-06-22T18:13:43.306 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash -- . 2023-06-22T18:13:43.348 INFO:teuthology.orchestra.run.smithi101.stderr:tar: /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-22T18:13:43.348 INFO:teuthology.orchestra.run.smithi101.stderr:tar: Error is not recoverable: exiting now 2023-06-22T18:13:43.350 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi133/crash 2023-06-22T18:13:43.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash -- . 2023-06-22T18:13:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-22T18:13:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2023-06-22T18:13:43.408 INFO:tasks.cephadm:Checking cluster log for badness... 2023-06-22T18:13:43.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-06-22T18:13:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:grep: /var/log/ceph/d572fea0-1126-11ee-9b2b-001a4aab830c/ceph.log: No such file or directory 2023-06-22T18:13:43.486 INFO:tasks.cephadm:Compressing logs... 2023-06-22T18:13:43.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T18:13:43.491 DEBUG:teuthology.orchestra.run.smithi101:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T18:13:43.494 DEBUG:teuthology.orchestra.run.smithi133:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T18:13:43.518 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-22T18:13:43.519 INFO:teuthology.orchestra.run.smithi101.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-22T18:13:43.548 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-22T18:13:44.757 INFO:tasks.cephadm:Archiving logs... 2023-06-22T18:13:44.758 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi039/log 2023-06-22T18:13:44.759 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-22T18:13:44.959 DEBUG:teuthology.misc:Transferring archived files from smithi101:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi101/log 2023-06-22T18:13:44.960 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-22T18:13:45.081 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi133/log 2023-06-22T18:13:45.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-22T18:13:45.187 INFO:tasks.cephadm:Removing cluster... 2023-06-22T18:13:45.187 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --force 2023-06-22T18:13:46.339 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --force 2023-06-22T18:13:47.126 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d572fea0-1126-11ee-9b2b-001a4aab830c --force 2023-06-22T18:13:47.909 INFO:tasks.cephadm:Removing cephadm ... 2023-06-22T18:13:47.910 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-22T18:13:47.953 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-22T18:13:47.970 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-22T18:13:47.984 INFO:tasks.cephadm:Teardown complete 2023-06-22T18:13:47.985 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1561, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1474, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 896, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 714, in create self.set_max_mds(max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 583, in set_max_mds self.set_var("max_mds", "%d" % max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5' 2023-06-22T18:13:48.078 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f24611e22d134360b355334d19dafd15 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1561, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 1474, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephadm.py", line 896, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 714, in create self.set_max_mds(max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 583, in set_max_mds self.set_var("max_mds", "%d" % max_mds) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5' 2023-06-22T18:13:48.084 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-06-22T18:13:48.101 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-22T18:13:48.147 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-22T18:13:48.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-22T18:13:48.165 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-22T18:13:48.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-22T18:13:48.303 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-22T18:13:48.317 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-22T18:13:48.317 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T18:13:48.322 DEBUG:teuthology.orchestra.run.smithi101:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T18:13:48.325 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-22T18:13:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-06-22T18:13:48.339 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-06-22T18:13:48.341 INFO:teuthology.orchestra.run.smithi133.stdout:210 Number of sources = 4 2023-06-22T18:13:48.342 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T18:13:48.342 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-06-22T18:13:48.342 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +1377us[+1377us] +/- 111ms 2023-06-22T18:13:48.342 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -2878us[-2643us] +/- 58ms 2023-06-22T18:13:48.342 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +2449us[+2684us] +/- 62ms 2023-06-22T18:13:48.343 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T18:13:48.343 INFO:teuthology.orchestra.run.smithi101.stdout:210 Number of sources = 4 2023-06-22T18:13:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T18:13:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-06-22T18:13:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +1365us[+1365us] +/- 111ms 2023-06-22T18:13:48.344 INFO:teuthology.orchestra.run.smithi101.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -2898us[-2653us] +/- 58ms 2023-06-22T18:13:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +2435us[+2679us] +/- 62ms 2023-06-22T18:13:48.345 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T18:13:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2023-06-22T18:13:48.359 INFO:teuthology.orchestra.run.smithi039.stdout:210 Number of sources = 4 2023-06-22T18:13:48.360 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-22T18:13:48.360 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2023-06-22T18:13:48.361 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +1668us[+1668us] +/- 111ms 2023-06-22T18:13:48.361 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv02.front.sepia.ceph.com 3 6 377 47 -2586us[-2572us] +/- 58ms 2023-06-22T18:13:48.362 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +2727us[+2727us] +/- 62ms 2023-06-22T18:13:48.362 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-22T18:13:48.364 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-22T18:13:48.373 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-22T18:13:48.374 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-22T18:13:48.406 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-22T18:13:48.569 DEBUG:teuthology.orchestra.run.smithi101:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-22T18:13:48.664 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-22T18:13:48.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T18:13:48.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T18:13:48.820 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T18:13:48.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T18:13:48.853 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-22T18:13:48.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T18:13:48.883 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-22T18:13:48.892 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-22T18:13:48.951 INFO:teuthology.task.internal:Duration was 1922.575111 seconds 2023-06-22T18:13:48.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-22T18:13:48.991 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-22T18:13:48.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-22T18:13:48.995 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-22T18:13:48.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-22T18:13:49.040 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T18:13:49.043 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T18:13:49.102 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-22T18:13:49.495 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-22T18:13:49.496 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2023-06-22T18:13:49.496 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-22T18:13:49.546 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi101.front.sepia.ceph.com 2023-06-22T18:13:49.547 DEBUG:teuthology.orchestra.run.smithi101:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-22T18:13:49.569 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2023-06-22T18:13:49.570 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-22T18:13:49.589 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-22T18:13:49.590 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T18:13:49.594 DEBUG:teuthology.orchestra.run.smithi101:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T18:13:49.612 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-22T18:13:49.675 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-22T18:13:49.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-22T18:13:49.680 DEBUG:teuthology.orchestra.run.smithi101:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-22T18:13:49.683 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-22T18:13:49.813 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-22T18:13:49.824 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-22T18:13:49.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-22T18:13:49.886 DEBUG:teuthology.orchestra.run.smithi101:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-22T18:13:49.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-22T18:13:49.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-22T18:13:49.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-22T18:13:49.950 DEBUG:teuthology.orchestra.run.smithi101:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-22T18:13:49.953 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-22T18:13:49.975 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = core 2023-06-22T18:13:49.998 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2023-06-22T18:13:50.006 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2023-06-22T18:13:50.053 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-22T18:13:50.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T18:13:50.087 DEBUG:teuthology.orchestra.run.smithi101:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-22T18:13:50.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T18:13:50.101 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-22T18:13:50.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-22T18:13:50.115 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-22T18:13:50.125 INFO:teuthology.task.internal:Transferring archived files... 2023-06-22T18:13:50.126 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi039 2023-06-22T18:13:50.126 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-22T18:13:50.199 DEBUG:teuthology.misc:Transferring archived files from smithi101:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi101 2023-06-22T18:13:50.200 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-22T18:13:50.233 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848/remote/smithi133 2023-06-22T18:13:50.234 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-22T18:13:50.266 INFO:teuthology.task.internal:Removing archive directory... 2023-06-22T18:13:50.267 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-22T18:13:50.271 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-22T18:13:50.274 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-22T18:13:50.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-22T18:13:50.332 INFO:teuthology.task.internal:Not uploading archives. 2023-06-22T18:13:50.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-22T18:13:50.357 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-22T18:13:50.358 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-22T18:13:50.362 DEBUG:teuthology.orchestra.run.smithi101:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-22T18:13:50.364 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-22T18:13:50.376 INFO:teuthology.orchestra.run.smithi101.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 22 18:13 /home/ubuntu/cephtest 2023-06-22T18:13:50.378 INFO:teuthology.orchestra.run.smithi133.stdout: 262149 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 22 18:13 /home/ubuntu/cephtest 2023-06-22T18:13:50.394 INFO:teuthology.orchestra.run.smithi039.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 22 18:13 /home/ubuntu/cephtest 2023-06-22T18:13:50.397 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-22T18:13:50.411 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-22T18:13:50.508 INFO:teuthology.nuke:Checking targets against current locks 2023-06-22T18:13:50.538 DEBUG:teuthology.nuke:shortname: smithi039 2023-06-22T18:13:50.539 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T18:13:50.565 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:30:57.946729', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDe9OFf/6dh5inwTSUjkG0HNiVExbLvI5xRjOsAWZcv8Kb/7u0naxIwsYjWC7YNUWCv44q3yyhAV/h/QxeBsEOaFH1azLrTZbMBDz6HIrr/vw4iDZQmVdTWSeyxmDpfcogvYfV5oMPVAnW6K/35J8gbEhI2mBAijc9rt5z4MKdc5jMmaMFhOmsTGkFqNMDg1jEwMjoFpHvfkWYEYhySYdmssODxy6u1vRwWUV3nizSTmowz+N2AhbLXwUkgkyEo1bMOHWimjcJqUGOisfoHIITTCjXXaOsxdxxsmQqHZd8kwcZC3p3RSzppg1htlEXnlkfT8JYvq1kTa4NMzGShv/zR+trwJIqiwNg5zQFS7TKRgpSPTvcdkzY1X7KmSAKJT2rLRDPzgy2qG0Sp1CKtTfoBEHRfjsghC4vuWkaUESIK66llnrzdYo/cFUNiqG0QsB+Spj2ZbGz9tyf/4/ig9gAI2C1Z5eyZnBbDc+Irx+xWn6giTO60IqJlL+/T9XMgO0M='} 2023-06-22T18:13:50.568 DEBUG:teuthology.nuke:shortname: smithi101 2023-06-22T18:13:50.569 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T18:13:50.594 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:30:57.944641', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjPijkdxgmpxaM2Hf+dEfbZJu1m48fjhao5gRK61VKJ4nYxXgoThOA3lR4BrmwCHdSiut3679K8GeK5t6yFBTU19ukf+irDuKwsyI7tmi3QrfhO83sedhLWgADiu9Jv+cDVUOFenhejI9ddgI4tzUXDo9g3RrQjEHqYBv1e8YBWcc3XZvnawpNxh1sn0VQX9+Y1z5baP0Yo8GT0pM5vNehWib1kMJVtVSWol/I+ZzYTVZR3SMZHYpAV4/uX+11mKKyBz9b8bj+ICDlaQI0ftLFGevk6ybr87UMG4SJEEpG1uYNMYJjEC4CZAHBFa5b9LlcjH7KVjpjS8mnYFTmA1PFoLAQVVwBx/A9D8MUydrZRfmYWp011IfXgryoQjTJWaorIhnGyE+5Ul7LZYxngmGmOnuMiS4YWzy7YeMEdc15Of2hFfMYnP0VAojJirAq2ILkKG23LCXSp2ypbceVuiImnDHbbXUSo/iiikoThKOTSXE6/il9xpKk/3BM26e69yU='} 2023-06-22T18:13:50.599 DEBUG:teuthology.nuke:shortname: smithi133 2023-06-22T18:13:50.599 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T18:13:50.624 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311848', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:30:57.948371', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDPW6QPBFRJ8SvhStxm6y5q211RlqOxyZcLIdkrKt2gUy9c3DZC0sn98VqvxjuSNV0U1mrc0olLrMsvfgfsMW0i0G2ELBGZURx+ynEO+MnCrp3kLat4P9qkT9rUkAqrYL6sy/jsLWh9DDX701oAZ6qqxAnOBpygcxthciZX//A9nalaoI/CZ1yvfsRaIawhN0HN9PanKHM3AoafR7v5YCMGLYDv16p1wB9spmY10sPMQB4NNf8R9DxlYo84DffMz/JbGzd5/HV6fg0HGJnPKvuOrfROWdvpCVmDXGbYdYL7CgUBs+KWss0qJj018ZUwL3r8aruv6E/oMxuYQnIqmipfaEm/RkRUH0CUFwMGMjkK814PfA+FBMYpn/weFm6ELBDywYeA8zJXiueBFCGSDEn6Q6EDHLf0d3LVIQ1Pmsi57jL7h7jyiQs6Rv/FlFkW68IJsr5UgWB4nrdiA953RMUoNJBSGaqoqg0kG8vgcUvw38t1VIFATIuP5jYRLNfOMcs='} 2023-06-22T18:13:50.633 INFO:teuthology.orchestra.console:Power off smithi039 2023-06-22T18:13:50.635 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T18:13:50.653 INFO:teuthology.orchestra.console:Power off smithi101 2023-06-22T18:13:50.654 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T18:13:50.669 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:50.684 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:50.709 INFO:teuthology.orchestra.console:Power off smithi133 2023-06-22T18:13:50.710 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T18:13:50.733 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:54.701 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:54.722 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:54.757 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:58.816 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:58.835 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:58.870 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:13:59.083 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2023-06-22T18:14:02.934 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:14:02.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:14:03.146 INFO:teuthology.orchestra.console:Power off for smithi039 completed 2023-06-22T18:14:03.160 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-06-22T18:14:03.265 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1c622d92b6f18ce44a7c33802686ff61c3c273ab description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/fs/misc} wsync/{yes}} duration: 1922.575110912323 failure_reason: 'Command failed on smithi039 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f24611e22d134360b355334d19dafd15 status: fail success: false 2023-06-22T18:14:03.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:14:03.354 INFO:teuthology.run:FAIL