2023-12-06T03:04:46.546 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-06T03:04:46.546 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} --owner scheduled_yuriw@teuthology --name yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi --verbose --archive /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630 -- /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/orig.config.yaml 2023-12-06T03:04:46.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:04:46.682 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630 branch: wip-yuri6-testing-2023-12-05-0753-pacific description: rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} email: yweinste@redhat.com first_in_suite: false job_id: '7478630' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/pacific v16.2.4 ' name: yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri6-testing-2023-12-05-0753-pacific ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2023-12-05-0753-pacific sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 5202 sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2023-12-05-0753-pacific suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b targets: smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDcA0LeUUrlefBBmmNRRUNrVs3nFmowXMKxFjHDEtrUlVsmfXtKHhFIpQjAilfPIkOAyAAeZ4HqdrBlG9BKkpB4= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTlPtw2z9JXlGBbJPqi4iTdHHOIH9ljBicL5hqG0DFhzKadshu9Kwap24oaWBhDZJ0D4laXPRyqYYmwe/5LXIQ= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - pexec: clients: - sudo modprobe -r ceph - sudo modprobe ceph disable_send_metrics=on - install: exclude_packages: - ceph-volume tag: v16.2.4 - print: '**** done install task...' - cephadm: avoid_pacific_features: true cephadm_branch: v16.2.4 cephadm_git_url: https://github.com/ceph/ceph image: docker.io/ceph/ceph:v16.2.4 roleless: true - print: '**** done starting v16.2.4' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-05_19:00:15 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2023-12-06T03:04:46.682 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa; will attempt to use it 2023-12-06T03:04:46.685 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks 2023-12-06T03:04:46.685 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-06T03:04:46.686 INFO:teuthology.task.internal:Checking packages... 2023-12-06T03:04:46.709 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd03fca4f35d43f61b4679936aef7bd2f01cced0b' 2023-12-06T03:04:46.709 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-06T03:04:46.709 INFO:teuthology.packaging:ref: None 2023-12-06T03:04:46.709 INFO:teuthology.packaging:tag: None 2023-12-06T03:04:46.709 INFO:teuthology.packaging:branch: wip-yuri6-testing-2023-12-05-0753-pacific 2023-12-06T03:04:46.709 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:04:46.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri6-testing-2023-12-05-0753-pacific 2023-12-06T03:04:47.008 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-335.gd03fca4f 2023-12-06T03:04:47.008 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-06T03:04:47.015 INFO:teuthology.task.internal:no buildpackages task found 2023-12-06T03:04:47.015 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-06T03:04:47.030 INFO:teuthology.task.internal:Saving configuration 2023-12-06T03:04:47.050 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-06T03:04:47.056 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T03:04:47.081 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 02:59:39.046450', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDcA0LeUUrlefBBmmNRRUNrVs3nFmowXMKxFjHDEtrUlVsmfXtKHhFIpQjAilfPIkOAyAAeZ4HqdrBlG9BKkpB4='} 2023-12-06T03:04:47.103 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 02:59:39.048003', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTlPtw2z9JXlGBbJPqi4iTdHHOIH9ljBicL5hqG0DFhzKadshu9Kwap24oaWBhDZJ0D4laXPRyqYYmwe/5LXIQ='} 2023-12-06T03:04:47.103 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-06T03:04:47.109 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2023-12-06T03:04:47.109 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2023-12-06T03:04:47.109 INFO:teuthology.run_tasks:Running task console_log... 2023-12-06T03:04:47.185 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f43c34e9a60>, signals=[15]) 2023-12-06T03:04:47.186 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-06T03:04:47.192 INFO:teuthology.task.internal:Opening connections... 2023-12-06T03:04:47.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2023-12-06T03:04:47.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:04:47.263 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2023-12-06T03:04:47.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:04:47.333 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-06T03:04:47.341 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2023-12-06T03:04:47.365 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2023-12-06T03:04:47.365 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2023-12-06T03:04:47.426 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-06T03:04:47.427 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T03:04:47.428 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2023-12-06T03:04:47.453 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2023-12-06T03:04:47.475 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2023-12-06T03:04:47.475 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2023-12-06T03:04:47.535 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-06T03:04:47.536 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T03:04:47.537 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2023-12-06T03:04:47.561 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-06T03:04:47.569 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-06T03:04:47.575 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-06T03:04:47.575 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2023-12-06T03:04:47.577 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2023-12-06T03:04:47.597 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-06T03:04:47.603 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-06T03:04:47.603 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2023-12-06T03:04:47.639 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2023-12-06T03:04:47.676 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-06T03:04:47.726 INFO:teuthology.run_tasks:Running task kernel... 2023-12-06T03:04:47.741 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-06T03:04:47.741 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-06T03:04:47.741 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-06T03:04:47.742 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-06T03:04:47.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:04:47.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:04:47.764 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2023-12-06T03:04:47.764 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2023-12-06T03:04:47.825 INFO:teuthology.orchestra.run.smithi055.stdout:4.18.0-489.el8.x86_64 2023-12-06T03:04:47.825 INFO:teuthology.task.kernel:Running kernel on smithi055: 4.18.0-489.el8.x86_64 2023-12-06T03:04:47.825 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum install -y kernel 2023-12-06T03:04:47.827 INFO:teuthology.orchestra.run.smithi175.stdout:4.18.0-489.el8.x86_64 2023-12-06T03:04:47.827 INFO:teuthology.task.kernel:Running kernel on smithi175: 4.18.0-489.el8.x86_64 2023-12-06T03:04:47.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2023-12-06T03:04:50.952 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - AppStream 52 MB/s | 34 MB 00:00 2023-12-06T03:04:51.228 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - AppStream 45 MB/s | 34 MB 00:00 2023-12-06T03:04:59.044 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - BaseOS 35 MB/s | 55 MB 00:01 2023-12-06T03:05:00.343 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - BaseOS 21 MB/s | 55 MB 00:02 2023-12-06T03:05:07.068 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - Extras 78 kB/s | 18 kB 00:00 2023-12-06T03:05:07.460 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - Extras common packages 32 kB/s | 6.9 kB 00:00 2023-12-06T03:05:07.770 INFO:teuthology.orchestra.run.smithi175.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-12-06T03:05:09.118 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - Extras 42 kB/s | 18 kB 00:00 2023-12-06T03:05:09.248 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-12-06T03:05:09.710 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - Extras common packages 18 kB/s | 6.9 kB 00:00 2023-12-06T03:05:10.062 INFO:teuthology.orchestra.run.smithi055.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-12-06T03:05:10.921 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-12-06T03:05:13.054 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 421 kB/s | 24 kB 00:00 2023-12-06T03:05:14.921 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 562 kB/s | 24 kB 00:00 2023-12-06T03:05:16.756 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-12-06T03:05:16.756 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-12-06T03:05:17.042 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:05:17.042 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:05:17.042 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:05:17.042 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:05:17.042 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 4.18.0-526.el8 baseos 10 M 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 4.18.0-526.el8 baseos 43 M 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 4.18.0-526.el8 baseos 36 M 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout:Install 3 Packages 2023-12-06T03:05:17.043 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:05:17.044 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 90 M 2023-12-06T03:05:17.044 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 96 M 2023-12-06T03:05:17.044 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:05:17.955 INFO:teuthology.orchestra.run.smithi175.stdout:(1/3): kernel-4.18.0-526.el8.x86_64.rpm 14 MB/s | 10 MB 00:00 2023-12-06T03:05:18.721 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-12-06T03:05:18.721 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-12-06T03:05:19.016 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:05:19.016 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout: kernel x86_64 4.18.0-526.el8 baseos 10 M 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core x86_64 4.18.0-526.el8 baseos 43 M 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules x86_64 4.18.0-526.el8 baseos 36 M 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout:Install 3 Packages 2023-12-06T03:05:19.017 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:05:19.018 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 90 M 2023-12-06T03:05:19.018 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 96 M 2023-12-06T03:05:19.018 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:05:19.347 INFO:teuthology.orchestra.run.smithi175.stdout:(2/3): kernel-core-4.18.0-526.el8.x86_64.rpm 21 MB/s | 43 MB 00:02 2023-12-06T03:05:19.681 INFO:teuthology.orchestra.run.smithi175.stdout:(3/3): kernel-modules-4.18.0-526.el8.x86_64.rpm 15 MB/s | 36 MB 00:02 2023-12-06T03:05:19.681 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:05:19.681 INFO:teuthology.orchestra.run.smithi175.stdout:Total 34 MB/s | 90 MB 00:02 2023-12-06T03:05:20.164 INFO:teuthology.orchestra.run.smithi055.stdout:(1/3): kernel-4.18.0-526.el8.x86_64.rpm 11 MB/s | 10 MB 00:00 2023-12-06T03:05:20.358 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:05:20.794 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:05:20.794 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:05:21.556 INFO:teuthology.orchestra.run.smithi055.stdout:(2/3): kernel-modules-4.18.0-526.el8.x86_64.rpm 15 MB/s | 36 MB 00:02 2023-12-06T03:05:22.032 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:05:22.054 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:05:22.198 INFO:teuthology.orchestra.run.smithi055.stdout:(3/3): kernel-core-4.18.0-526.el8.x86_64.rpm 15 MB/s | 43 MB 00:02 2023-12-06T03:05:22.198 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:05:22.198 INFO:teuthology.orchestra.run.smithi055.stdout:Total 28 MB/s | 90 MB 00:03 2023-12-06T03:05:22.905 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:05:23.368 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:05:23.368 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:05:24.677 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:05:24.702 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:05:24.864 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:05:25.345 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-4.18.0-526.el8.x86_64 1/3 2023-12-06T03:05:27.663 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:05:28.200 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-core-4.18.0-526.el8.x86_64 1/3 2023-12-06T03:05:28.922 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 1/3 2023-12-06T03:05:29.520 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-12-06T03:05:31.951 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 1/3 2023-12-06T03:05:32.358 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-12-06T03:05:32.949 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-12-06T03:05:33.224 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:36.029 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 2/3 2023-12-06T03:05:36.292 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:55.466 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:55.490 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:55.827 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:55.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-4.18.0-526.el8.x86_64 1/3 2023-12-06T03:05:55.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-4.18.0-526.el8.x86_64 2/3 2023-12-06T03:05:56.577 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:56.577 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:05:56.577 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:05:56.577 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-4.18.0-526.el8.x86_64 kernel-core-4.18.0-526.el8.x86_64 2023-12-06T03:05:56.577 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-4.18.0-526.el8.x86_64 2023-12-06T03:05:56.578 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:05:56.578 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:05:56.946 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T03:05:57.161 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-4.18.0-526.el8.x86_64 2023-12-06T03:05:57.162 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-12-06T03:05:57.162 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-526.el8.x86_64 2023-12-06T03:05:57.162 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-06T03:05:57.162 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-06T03:05:57.162 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2023-12-06T03:05:57.162 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2023-12-06T03:05:57.162 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T03:05:57.372 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-4.18.0-526.el8.x86_64 2023-12-06T03:05:57.373 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-12-06T03:05:57.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.02 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 129.el8 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Group : System Environment/Base 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 9265586 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-12-06T03:05:57.407 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : kbuilder.bsys.centos.org 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Relocations : (not relocatable) 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : CentOS BuildSystem 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-06T03:05:57.408 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-06T03:05:57.409 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-06T03:05:57.409 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2023-12-06T03:05:57.409 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:05:57.409 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2023-12-06T03:05:57.410 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-06T03:05:57.410 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 4.18.0-526.el8.x86_64 2023-12-06T03:05:57.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-06T03:05:58.812 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2023-12-06T03:05:59.711 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:05:59.735 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:06:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:06:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-4.18.0-526.el8.x86_64 1/3 2023-12-06T03:06:00.102 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-core-4.18.0-526.el8.x86_64 2/3 2023-12-06T03:06:00.900 INFO:teuthology.orchestra.run.smithi175.stderr:done 2023-12-06T03:06:00.901 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2023-12-06T03:06:00.924 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.s9eIZ7MS3q 2023-12-06T03:06:00.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.s9eIZ7MS3q 2023-12-06T03:06:00.991 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-4.18.0-526.el8.x86_64 3/3 2023-12-06T03:06:00.991 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:06:00.991 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:06:00.991 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-4.18.0-526.el8.x86_64 kernel-core-4.18.0-526.el8.x86_64 2023-12-06T03:06:00.991 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-4.18.0-526.el8.x86_64 2023-12-06T03:06:00.991 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:06:00.992 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:06:01.001 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.s9eIZ7MS3q 2023-12-06T03:06:01.142 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T03:06:01.154 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.s9eIZ7MS3q is 6KB 2023-12-06T03:06:01.202 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.s9eIZ7MS3q 2023-12-06T03:06:01.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2023-12-06T03:06:01.284 INFO:teuthology.orchestra.run.smithi175.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-12-06T03:06:01.284 INFO:teuthology.orchestra.run.smithi175.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-12-06T03:06:01.284 INFO:teuthology.orchestra.run.smithi175.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-12-06T03:06:01.285 INFO:teuthology.orchestra.run.smithi175.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64.conf 2023-12-06T03:06:01.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64 2023-12-06T03:06:01.366 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-4.18.0-526.el8.x86_64 2023-12-06T03:06:01.367 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-12-06T03:06:01.367 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-526.el8.x86_64 2023-12-06T03:06:01.367 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-06T03:06:01.367 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-06T03:06:01.367 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2023-12-06T03:06:01.367 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=None, version=distro) 2023-12-06T03:06:01.367 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T03:06:01.446 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2023-12-06T03:06:01.586 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-4.18.0-526.el8.x86_64 2023-12-06T03:06:01.587 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-526.el8.x86_64 2023-12-06T03:06:01.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2023-12-06T03:06:01.622 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2023-12-06T03:06:01.622 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2023-12-06T03:06:01.622 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.02 2023-12-06T03:06:01.622 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 129.el8 2023-12-06T03:06:01.622 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Group : System Environment/Base 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 9265586 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : kbuilder.bsys.centos.org 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Relocations : (not relocatable) 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : CentOS BuildSystem 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : CentOS 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2023-12-06T03:06:01.623 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:06:01.624 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2023-12-06T03:06:01.625 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-06T03:06:01.625 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 4.18.0-526.el8.x86_64 2023-12-06T03:06:01.625 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-06T03:06:02.996 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2023-12-06T03:06:05.168 INFO:teuthology.orchestra.run.smithi055.stderr:done 2023-12-06T03:06:05.179 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2023-12-06T03:06:05.208 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.9R9uuWqsAM 2023-12-06T03:06:05.208 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9R9uuWqsAM 2023-12-06T03:06:05.280 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.9R9uuWqsAM 2023-12-06T03:06:05.436 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.9R9uuWqsAM is 6KB 2023-12-06T03:06:05.483 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.9R9uuWqsAM 2023-12-06T03:06:05.501 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2023-12-06T03:06:05.573 INFO:teuthology.orchestra.run.smithi055.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-12-06T03:06:05.573 INFO:teuthology.orchestra.run.smithi055.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-12-06T03:06:05.574 INFO:teuthology.orchestra.run.smithi055.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-12-06T03:06:05.574 INFO:teuthology.orchestra.run.smithi055.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64.conf 2023-12-06T03:06:05.575 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-526.el8.x86_64 2023-12-06T03:06:05.760 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2023-12-06T03:06:31.468 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2023-12-06T03:06:31.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-12-06T03:06:31.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:06:35.767 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2023-12-06T03:06:35.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-06T03:06:35.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:06:54.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-06T03:07:03.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-06T03:07:03.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:07:21.570 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2023-12-06T03:07:31.478 DEBUG:teuthology.orchestra.remote:timed out 2023-12-06T03:07:33.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-06T03:07:33.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:07:34.789 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-12-06T03:07:35.128 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-06T03:07:35.128 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-526.el8.x86_64"... 2023-12-06T03:07:35.128 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2023-12-06T03:07:35.150 INFO:teuthology.orchestra.run.smithi055.stdout:4.18.0-526.el8.x86_64 2023-12-06T03:07:35.151 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-526.el8.x86_64 vs 4.18.0-526.el8.x86_64 2023-12-06T03:07:35.151 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-06T03:07:35.151 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-06T03:07:36.152 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2023-12-06T03:07:36.152 DEBUG:teuthology.orchestra.run.smithi055:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-06T03:07:36.221 INFO:teuthology.orchestra.run.smithi055.stdout:ttyS1 2023-12-06T03:07:36.237 DEBUG:teuthology.parallel:result is None 2023-12-06T03:07:40.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-12-06T03:07:40.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:07:41.014 DEBUG:teuthology.orchestra.run.smithi175:> true 2023-12-06T03:07:41.370 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-12-06T03:07:41.370 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-526.el8.x86_64"... 2023-12-06T03:07:41.370 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2023-12-06T03:07:41.387 INFO:teuthology.orchestra.run.smithi175.stdout:4.18.0-526.el8.x86_64 2023-12-06T03:07:41.387 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-526.el8.x86_64 vs 4.18.0-526.el8.x86_64 2023-12-06T03:07:41.388 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-06T03:07:41.388 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-06T03:07:42.389 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2023-12-06T03:07:42.389 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-06T03:07:42.469 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2023-12-06T03:07:42.481 DEBUG:teuthology.parallel:result is None 2023-12-06T03:07:42.481 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-06T03:07:42.488 INFO:teuthology.task.internal:Creating test directory... 2023-12-06T03:07:42.489 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-06T03:07:42.493 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-06T03:07:42.514 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-06T03:07:42.522 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-06T03:07:42.532 INFO:teuthology.task.internal:Creating archive directory... 2023-12-06T03:07:42.532 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-06T03:07:42.554 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-06T03:07:42.585 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-06T03:07:42.593 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-06T03:07:42.593 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-06T03:07:42.620 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-06T03:07:42.666 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T03:07:42.676 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T03:07:42.780 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T03:07:42.794 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T03:07:42.796 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-06T03:07:42.803 INFO:teuthology.task.internal:Configuring sudo... 2023-12-06T03:07:42.803 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-06T03:07:42.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-06T03:07:42.837 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-06T03:07:42.851 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-06T03:07:42.851 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-06T03:07:42.877 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-06T03:07:42.896 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T03:07:42.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T03:07:43.071 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T03:07:43.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T03:07:43.194 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:07:43.194 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-06T03:07:43.622 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T03:07:43.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T03:07:43.714 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T03:07:43.797 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T03:07:43.838 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:07:43.838 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-06T03:07:43.905 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2023-12-06T03:07:43.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2023-12-06T03:07:44.019 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T03:07:44.666 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T03:07:44.777 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-06T03:07:44.785 INFO:teuthology.task.internal:Starting timer... 2023-12-06T03:07:44.785 INFO:teuthology.run_tasks:Running task pcp... 2023-12-06T03:07:44.802 INFO:teuthology.run_tasks:Running task selinux... 2023-12-06T03:07:44.815 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-12-06T03:07:44.815 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2023-12-06T03:07:44.954 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: 2023-12-06T03:07:44.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2023-12-06T03:07:45.053 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2023-12-06T03:07:45.055 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-06T03:07:45.055 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2023-12-06T03:07:45.195 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2023-12-06T03:07:45.195 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2023-12-06T03:07:45.223 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2023-12-06T03:07:45.224 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2023-12-06T03:07:45.224 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T03:07:45.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:07:45.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T03:07:45.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:07:45.295 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-06T03:07:45.295 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2023-12-06T03:07:45.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2023-12-06T03:07:45.419 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-06T03:07:45.429 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-06T03:07:45.429 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-06T03:07:45.445 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-06T03:07:45.446 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-06T03:16:03.993 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2023-12-06T03:16:03.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-06T03:16:03.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:16:04.068 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-12-06T03:16:04.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-12-06T03:16:04.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-12-06T03:16:04.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T03:16:04.218 DEBUG:teuthology.orchestra.run.smithi175:> true 2023-12-06T03:16:04.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2023-12-06T03:16:04.289 INFO:teuthology.run_tasks:Running task clock... 2023-12-06T03:16:04.299 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-06T03:16:04.299 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-06T03:16:04.300 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T03:16:04.303 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-06T03:16:04.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T03:16:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-06T03:16:04.358 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-06T03:16:04.362 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-06T03:16:04.376 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-06T03:16:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2023-12-06T03:16:04.441 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2023-12-06T03:16:04.449 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2023-12-06T03:16:04.458 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-06T03:16:04.460 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2023-12-06T03:16:04.473 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-06T03:16:04.474 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-06T03:16:04.487 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-06T03:16:04.547 INFO:teuthology.orchestra.run.smithi055.stderr:bash: ntpq: command not found 2023-12-06T03:16:04.549 INFO:teuthology.orchestra.run.smithi175.stderr:bash: ntpq: command not found 2023-12-06T03:16:04.551 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T03:16:04.551 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2023-12-06T03:16:04.551 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.552 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.552 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.552 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.554 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T03:16:04.554 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-12-06T03:16:04.554 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.554 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.555 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.555 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:16:04.555 INFO:teuthology.run_tasks:Running task pexec... 2023-12-06T03:16:04.569 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-06T03:16:04.569 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi055.front.sepia.ceph.com 2023-12-06T03:16:04.570 DEBUG:teuthology.orchestra.run.smithi055:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-06T03:16:04.570 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi175.front.sepia.ceph.com 2023-12-06T03:16:04.570 DEBUG:teuthology.orchestra.run.smithi175:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-06T03:16:05.473 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:23 ago on Wed 06 Dec 2023 03:13:42 AM UTC. 2023-12-06T03:16:05.606 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:42 ago on Wed 06 Dec 2023 03:14:23 AM UTC. 2023-12-06T03:16:07.595 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout:Resetting modules: 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout: container-tools 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:07.596 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:07.841 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:16:07.994 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout:Resetting modules: 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout: container-tools 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:07.998 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:16:07.999 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:07.999 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:08.295 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:16:08.793 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:26 ago on Wed 06 Dec 2023 03:13:42 AM UTC. 2023-12-06T03:16:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:46 ago on Wed 06 Dec 2023 03:14:23 AM UTC. 2023-12-06T03:16:10.868 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:16:11.052 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:11.052 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout:Installing group/module packages: 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: aardvark-dns x86_64 2:1.8.0-1.module_el8+712+4cd1bd69 appstream 1.0 M 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: buildah x86_64 1:1.32.0-1.module_el8+712+4cd1bd69 appstream 9.6 M 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: cockpit-podman noarch 78-1.module_el8+712+4cd1bd69 appstream 750 k 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: crun x86_64 1.9.2-1.module_el8+712+4cd1bd69 appstream 241 k 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: netavark x86_64 2:1.8.0-2.module_el8+712+4cd1bd69 appstream 3.9 M 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: python3-podman noarch 4.7.0-1.module_el8+712+4cd1bd69 appstream 178 k 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: skopeo x86_64 2:1.13.3-1.module_el8+712+4cd1bd69 appstream 8.1 M 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: toolbox x86_64 0.0.99.4-6.module_el8+712+4cd1bd69 appstream 2.5 M 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: udica noarch 0.2.6-20.module_el8+712+4cd1bd69 appstream 53 k 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2023-12-06T03:16:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout:Installing module profiles: 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: container-tools/common 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout:Enabling module streams: 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: container-tools rhel8 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout:Install 13 Packages 2023-12-06T03:16:11.054 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:11.055 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 27 M 2023-12-06T03:16:11.055 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 89 M 2023-12-06T03:16:11.055 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:16:11.373 INFO:teuthology.orchestra.run.smithi175.stdout:(1/13): cockpit-podman-78-1.module_el8+712+4cd1 6.9 MB/s | 750 kB 00:00 2023-12-06T03:16:11.432 INFO:teuthology.orchestra.run.smithi175.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 649 kB/s | 38 kB 00:00 2023-12-06T03:16:11.465 INFO:teuthology.orchestra.run.smithi175.stdout:(3/13): aardvark-dns-1.8.0-1.module_el8+712+4cd 5.1 MB/s | 1.0 MB 00:00 2023-12-06T03:16:11.491 INFO:teuthology.orchestra.run.smithi175.stdout:(4/13): crun-1.9.2-1.module_el8+712+4cd1bd69.x8 4.0 MB/s | 241 kB 00:00 2023-12-06T03:16:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:(5/13): buildah-1.32.0-1.module_el8+712+4cd1bd6 29 MB/s | 9.6 MB 00:00 2023-12-06T03:16:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:(6/13): python3-podman-4.7.0-1.module_el8+712+4 1.3 MB/s | 178 kB 00:00 2023-12-06T03:16:11.705 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:16:11.717 INFO:teuthology.orchestra.run.smithi175.stdout:(7/13): netavark-1.8.0-2.module_el8+712+4cd1bd6 16 MB/s | 3.9 MB 00:00 2023-12-06T03:16:11.742 INFO:teuthology.orchestra.run.smithi175.stdout:(8/13): python3-pytoml-0.1.14-5.git7dea353.el8. 174 kB/s | 25 kB 00:00 2023-12-06T03:16:11.767 INFO:teuthology.orchestra.run.smithi175.stdout:(9/13): python3-pyxdg-0.25-16.el8.noarch.rpm 660 kB/s | 94 kB 00:00 2023-12-06T03:16:11.901 INFO:teuthology.orchestra.run.smithi175.stdout:(10/13): skopeo-1.13.3-1.module_el8+712+4cd1bd6 44 MB/s | 8.1 MB 00:00 2023-12-06T03:16:11.910 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout:Installing group/module packages: 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: aardvark-dns x86_64 2:1.8.0-1.module_el8+712+4cd1bd69 appstream 1.0 M 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: buildah x86_64 1:1.32.0-1.module_el8+712+4cd1bd69 appstream 9.6 M 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: cockpit-podman noarch 78-1.module_el8+712+4cd1bd69 appstream 750 k 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: crun x86_64 1.9.2-1.module_el8+712+4cd1bd69 appstream 241 k 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: netavark x86_64 2:1.8.0-2.module_el8+712+4cd1bd69 appstream 3.9 M 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: python3-podman noarch 4.7.0-1.module_el8+712+4cd1bd69 appstream 178 k 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: skopeo x86_64 2:1.13.3-1.module_el8+712+4cd1bd69 appstream 8.1 M 2023-12-06T03:16:11.911 INFO:teuthology.orchestra.run.smithi055.stdout: toolbox x86_64 0.0.99.4-6.module_el8+712+4cd1bd69 appstream 2.5 M 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: udica noarch 0.2.6-20.module_el8+712+4cd1bd69 appstream 53 k 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout:Installing weak dependencies: 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout:Installing module profiles: 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: container-tools/common 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling module streams: 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: container-tools rhel8 2023-12-06T03:16:11.912 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout:Install 13 Packages 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 27 M 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 89 M 2023-12-06T03:16:11.913 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:16:11.926 INFO:teuthology.orchestra.run.smithi175.stdout:(11/13): udica-0.2.6-20.module_el8+712+4cd1bd69 332 kB/s | 53 kB 00:00 2023-12-06T03:16:11.952 INFO:teuthology.orchestra.run.smithi175.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 820 kB/s | 41 kB 00:00 2023-12-06T03:16:12.002 INFO:teuthology.orchestra.run.smithi175.stdout:(13/13): toolbox-0.0.99.4-6.module_el8+712+4cd1 9.8 MB/s | 2.5 MB 00:00 2023-12-06T03:16:12.002 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:16:12.002 INFO:teuthology.orchestra.run.smithi175.stdout:Total 28 MB/s | 27 MB 00:00 2023-12-06T03:16:12.053 INFO:teuthology.orchestra.run.smithi055.stdout:(1/13): cockpit-podman-78-1.module_el8+712+4cd1 7.7 MB/s | 750 kB 00:00 2023-12-06T03:16:12.086 INFO:teuthology.orchestra.run.smithi055.stdout:(2/13): aardvark-dns-1.8.0-1.module_el8+712+4cd 7.8 MB/s | 1.0 MB 00:00 2023-12-06T03:16:12.103 INFO:teuthology.orchestra.run.smithi055.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 758 kB/s | 38 kB 00:00 2023-12-06T03:16:12.129 INFO:teuthology.orchestra.run.smithi055.stdout:(4/13): crun-1.9.2-1.module_el8+712+4cd1bd69.x8 5.6 MB/s | 241 kB 00:00 2023-12-06T03:16:12.162 INFO:teuthology.orchestra.run.smithi055.stdout:(5/13): python3-podman-4.7.0-1.module_el8+712+4 5.2 MB/s | 178 kB 00:00 2023-12-06T03:16:12.196 INFO:teuthology.orchestra.run.smithi055.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 750 kB/s | 25 kB 00:00 2023-12-06T03:16:12.230 INFO:teuthology.orchestra.run.smithi055.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 2.8 MB/s | 94 kB 00:00 2023-12-06T03:16:12.270 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:16:12.288 INFO:teuthology.orchestra.run.smithi055.stdout:(8/13): netavark-1.8.0-2.module_el8+712+4cd1bd6 21 MB/s | 3.9 MB 00:00 2023-12-06T03:16:12.291 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:16:12.291 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:16:12.389 INFO:teuthology.orchestra.run.smithi055.stdout:(9/13): toolbox-0.0.99.4-6.module_el8+712+4cd1b 25 MB/s | 2.5 MB 00:00 2023-12-06T03:16:12.497 INFO:teuthology.orchestra.run.smithi055.stdout:(10/13): buildah-1.32.0-1.module_el8+712+4cd1bd 18 MB/s | 9.6 MB 00:00 2023-12-06T03:16:12.522 INFO:teuthology.orchestra.run.smithi055.stdout:(11/13): udica-0.2.6-20.module_el8+712+4cd1bd69 394 kB/s | 53 kB 00:00 2023-12-06T03:16:12.542 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:16:12.546 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:16:12.547 INFO:teuthology.orchestra.run.smithi055.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 819 kB/s | 41 kB 00:00 2023-12-06T03:16:12.656 INFO:teuthology.orchestra.run.smithi055.stdout:(13/13): skopeo-1.13.3-1.module_el8+712+4cd1bd6 19 MB/s | 8.1 MB 00:00 2023-12-06T03:16:12.657 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:16:12.657 INFO:teuthology.orchestra.run.smithi055.stdout:Total 36 MB/s | 27 MB 00:00 2023-12-06T03:16:12.949 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:16:12.973 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:16:12.973 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:16:13.227 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:16:13.262 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:16:13.266 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:16:13.453 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2023-12-06T03:16:13.649 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-06T03:16:13.840 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/13 2023-12-06T03:16:14.076 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2023-12-06T03:16:14.078 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:16:14.315 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2023-12-06T03:16:14.457 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-06T03:16:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : aardvark-dns-2:1.8.0-1.module_el8+712+4cd1bd69.x86 5/13 2023-12-06T03:16:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/13 2023-12-06T03:16:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2023-12-06T03:16:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : aardvark-dns-2:1.8.0-1.module_el8+712+4cd1bd69.x86 5/13 2023-12-06T03:16:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : netavark-2:1.8.0-2.module_el8+712+4cd1bd69.x86_64 6/13 2023-12-06T03:16:15.854 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : buildah-1:1.32.0-1.module_el8+712+4cd1bd69.x86_64 7/13 2023-12-06T03:16:16.051 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : crun-1.9.2-1.module_el8+712+4cd1bd69.x86_64 8/13 2023-12-06T03:16:16.327 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-podman-4.7.0-1.module_el8+712+4cd1bd69.noa 9/13 2023-12-06T03:16:16.549 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : netavark-2:1.8.0-2.module_el8+712+4cd1bd69.x86_64 6/13 2023-12-06T03:16:16.693 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : udica-0.2.6-20.module_el8+712+4cd1bd69.noarch 10/13 2023-12-06T03:16:16.798 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : buildah-1:1.32.0-1.module_el8+712+4cd1bd69.x86_64 7/13 2023-12-06T03:16:16.999 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : crun-1.9.2-1.module_el8+712+4cd1bd69.x86_64 8/13 2023-12-06T03:16:17.263 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-podman-4.7.0-1.module_el8+712+4cd1bd69.noa 9/13 2023-12-06T03:16:17.573 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : toolbox-0.0.99.4-6.module_el8+712+4cd1bd69.x86_64 11/13 2023-12-06T03:16:17.706 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : udica-0.2.6-20.module_el8+712+4cd1bd69.noarch 10/13 2023-12-06T03:16:17.807 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : skopeo-2:1.13.3-1.module_el8+712+4cd1bd69.x86_64 12/13 2023-12-06T03:16:18.005 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 13/13 2023-12-06T03:16:18.451 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 13/13 2023-12-06T03:16:18.451 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : aardvark-dns-2:1.8.0-1.module_el8+712+4cd1bd69.x86 1/13 2023-12-06T03:16:18.451 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : buildah-1:1.32.0-1.module_el8+712+4cd1bd69.x86_64 2/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 3/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : crun-1.9.2-1.module_el8+712+4cd1bd69.x86_64 5/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : netavark-2:1.8.0-2.module_el8+712+4cd1bd69.x86_64 6/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-podman-4.7.0-1.module_el8+712+4cd1bd69.noa 7/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : skopeo-2:1.13.3-1.module_el8+712+4cd1bd69.x86_64 10/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : toolbox-0.0.99.4-6.module_el8+712+4cd1bd69.x86_64 11/13 2023-12-06T03:16:18.452 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : udica-0.2.6-20.module_el8+712+4cd1bd69.noarch 12/13 2023-12-06T03:16:18.779 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : toolbox-0.0.99.4-6.module_el8+712+4cd1bd69.x86_64 11/13 2023-12-06T03:16:19.032 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : skopeo-2:1.13.3-1.module_el8+712+4cd1bd69.x86_64 12/13 2023-12-06T03:16:19.214 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 13/13 2023-12-06T03:16:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2023-12-06T03:16:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: aardvark-dns-2:1.8.0-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: buildah-1:1.32.0-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: crun-1.9.2-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: netavark-2:1.8.0-2.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: python3-podman-4.7.0-1.module_el8+712+4cd1bd69.noarch 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: skopeo-2:1.13.3-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: toolbox-0.0.99.4-6.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: udica-0.2.6-20.module_el8+712+4cd1bd69.noarch 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-06T03:16:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:19.272 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:16:19.683 DEBUG:teuthology.parallel:result is None 2023-12-06T03:16:19.685 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 13/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : aardvark-dns-2:1.8.0-1.module_el8+712+4cd1bd69.x86 1/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : buildah-1:1.32.0-1.module_el8+712+4cd1bd69.x86_64 2/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 3/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : crun-1.9.2-1.module_el8+712+4cd1bd69.x86_64 5/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : netavark-2:1.8.0-2.module_el8+712+4cd1bd69.x86_64 6/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-podman-4.7.0-1.module_el8+712+4cd1bd69.noa 7/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : skopeo-2:1.13.3-1.module_el8+712+4cd1bd69.x86_64 10/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : toolbox-0.0.99.4-6.module_el8+712+4cd1bd69.x86_64 11/13 2023-12-06T03:16:19.686 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : udica-0.2.6-20.module_el8+712+4cd1bd69.noarch 12/13 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: aardvark-dns-2:1.8.0-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: buildah-1:1.32.0-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: cockpit-podman-78-1.module_el8+712+4cd1bd69.noarch 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: crun-1.9.2-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: netavark-2:1.8.0-2.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: python3-podman-4.7.0-1.module_el8+712+4cd1bd69.noarch 2023-12-06T03:16:20.586 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: skopeo-2:1.13.3-1.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: toolbox-0.0.99.4-6.module_el8+712+4cd1bd69.x86_64 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: udica-0.2.6-20.module_el8+712+4cd1bd69.noarch 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:20.587 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:16:20.834 DEBUG:teuthology.parallel:result is None 2023-12-06T03:16:20.837 INFO:teuthology.run_tasks:Running task pexec... 2023-12-06T03:16:20.846 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-06T03:16:20.847 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi055.front.sepia.ceph.com 2023-12-06T03:16:20.847 DEBUG:teuthology.orchestra.run.smithi055:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-06T03:16:20.848 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi175.front.sepia.ceph.com 2023-12-06T03:16:20.848 DEBUG:teuthology.orchestra.run.smithi175:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-06T03:16:21.002 DEBUG:teuthology.parallel:result is None 2023-12-06T03:16:21.059 DEBUG:teuthology.parallel:result is None 2023-12-06T03:16:21.059 INFO:teuthology.run_tasks:Running task install... 2023-12-06T03:16:21.070 DEBUG:teuthology.task.install:project ceph 2023-12-06T03:16:21.071 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd03fca4f35d43f61b4679936aef7bd2f01cced0b'}} 2023-12-06T03:16:21.071 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v16.2.4', 'flavor': 'default', 'sha1': 'd03fca4f35d43f61b4679936aef7bd2f01cced0b'} 2023-12-06T03:16:21.071 INFO:teuthology.task.install:Using flavor: default 2023-12-06T03:16:21.078 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-06T03:16:21.078 INFO:teuthology.task.install:extra packages: [] 2023-12-06T03:16:21.079 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd03fca4f35d43f61b4679936aef7bd2f01cced0b', 'tag': 'v16.2.4', 'wait_for_package': False} 2023-12-06T03:16:21.079 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2023-12-06T03:16:21.079 INFO:teuthology.packaging:ref: None 2023-12-06T03:16:21.080 INFO:teuthology.packaging:tag: v16.2.4 2023-12-06T03:16:21.080 INFO:teuthology.packaging:branch: None 2023-12-06T03:16:21.080 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:16:21.317 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:16:21.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:16:21.320 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd03fca4f35d43f61b4679936aef7bd2f01cced0b', 'tag': 'v16.2.4', 'wait_for_package': False} 2023-12-06T03:16:21.320 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2023-12-06T03:16:21.320 INFO:teuthology.packaging:ref: None 2023-12-06T03:16:21.320 INFO:teuthology.packaging:tag: v16.2.4 2023-12-06T03:16:21.320 INFO:teuthology.packaging:branch: None 2023-12-06T03:16:21.320 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:16:21.557 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:16:21.558 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:16:21.777 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2023-12-06T03:16:21.777 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2023-12-06T03:16:21.801 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2023-12-06T03:16:21.802 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2023-12-06T03:16:21.825 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-06T03:16:21.826 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:16:21.826 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-06T03:16:21.842 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-06T03:16:21.842 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:16:21.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-06T03:16:21.866 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-12-06T03:16:21.866 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2023-12-06T03:16:21.866 INFO:teuthology.packaging:ref: None 2023-12-06T03:16:21.866 INFO:teuthology.packaging:tag: v16.2.4 2023-12-06T03:16:21.866 INFO:teuthology.packaging:branch: None 2023-12-06T03:16:21.866 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:16:21.866 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-06T03:16:21.876 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-12-06T03:16:21.876 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2023-12-06T03:16:21.876 INFO:teuthology.packaging:ref: None 2023-12-06T03:16:21.876 INFO:teuthology.packaging:tag: v16.2.4 2023-12-06T03:16:21.877 INFO:teuthology.packaging:branch: None 2023-12-06T03:16:21.877 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:16:21.877 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-06T03:16:21.943 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-06T03:16:21.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-06T03:16:22.028 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-06T03:16:22.030 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-06T03:16:22.062 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2023-12-06T03:16:22.064 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2023-12-06T03:16:22.096 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2023-12-06T03:16:22.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2023-12-06T03:16:22.379 INFO:teuthology.orchestra.run.smithi175.stdout:76 files removed 2023-12-06T03:16:22.379 INFO:teuthology.orchestra.run.smithi055.stdout:76 files removed 2023-12-06T03:16:22.408 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw 2023-12-06T03:16:22.414 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw 2023-12-06T03:16:22.895 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 918 kB/s | 74 kB 00:00 2023-12-06T03:16:22.965 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 928 kB/s | 74 kB 00:00 2023-12-06T03:16:23.038 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 145 kB/s | 15 kB 00:00 2023-12-06T03:16:23.082 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 203 kB/s | 15 kB 00:00 2023-12-06T03:16:23.163 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 18 kB/s | 1.7 kB 00:00 2023-12-06T03:16:23.333 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 8.0 kB/s | 1.7 kB 00:00 2023-12-06T03:16:23.805 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - AppStream 56 MB/s | 34 MB 00:00 2023-12-06T03:16:24.192 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - AppStream 41 MB/s | 34 MB 00:00 2023-12-06T03:16:32.206 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - BaseOS 26 MB/s | 55 MB 00:02 2023-12-06T03:16:33.268 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - BaseOS 26 MB/s | 55 MB 00:02 2023-12-06T03:16:40.297 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - Extras 73 kB/s | 18 kB 00:00 2023-12-06T03:16:40.697 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - Extras common packages 31 kB/s | 6.9 kB 00:00 2023-12-06T03:16:41.482 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 8 - PowerTools 10 MB/s | 6.6 MB 00:00 2023-12-06T03:16:42.301 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - Extras 73 kB/s | 18 kB 00:00 2023-12-06T03:16:42.735 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - Extras common packages 30 kB/s | 6.9 kB 00:00 2023-12-06T03:16:42.975 INFO:teuthology.orchestra.run.smithi175.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-12-06T03:16:43.520 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 8 - PowerTools 11 MB/s | 6.6 MB 00:00 2023-12-06T03:16:44.937 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 16 MB 00:01 2023-12-06T03:16:45.130 INFO:teuthology.orchestra.run.smithi055.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-12-06T03:16:46.947 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2023-12-06T03:16:48.759 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 429 kB/s | 24 kB 00:00 2023-12-06T03:16:51.273 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 335 kB/s | 24 kB 00:00 2023-12-06T03:16:53.344 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:16:53.344 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: daxctl-libs x86_64 71.1-7.el8 baseos 42 k 2023-12-06T03:16:53.345 INFO:teuthology.orchestra.run.smithi175.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.6.1-1.el8 appstream 145 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: ndctl-libs x86_64 71.1-7.el8 baseos 80 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2023-12-06T03:16:53.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout:Install 25 Packages 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2023-12-06T03:16:53.347 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:16:53.348 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 59 M 2023-12-06T03:16:53.348 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:16:53.994 INFO:teuthology.orchestra.run.smithi175.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 44 MB/s | 6.0 MB 00:00 2023-12-06T03:16:54.019 INFO:teuthology.orchestra.run.smithi175.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 995 kB/s | 25 kB 00:00 2023-12-06T03:16:54.053 INFO:teuthology.orchestra.run.smithi175.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 24 MB/s | 801 kB 00:00 2023-12-06T03:16:54.086 INFO:teuthology.orchestra.run.smithi175.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 14 MB/s | 486 kB 00:00 2023-12-06T03:16:54.187 INFO:teuthology.orchestra.run.smithi175.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 41 MB/s | 4.1 MB 00:00 2023-12-06T03:16:54.212 INFO:teuthology.orchestra.run.smithi175.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 1.7 MB/s | 44 kB 00:00 2023-12-06T03:16:54.237 INFO:teuthology.orchestra.run.smithi175.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 3.1 MB/s | 80 kB 00:00 2023-12-06T03:16:54.262 INFO:teuthology.orchestra.run.smithi175.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 8.3 MB/s | 213 kB 00:00 2023-12-06T03:16:54.296 INFO:teuthology.orchestra.run.smithi175.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 11 MB/s | 387 kB 00:00 2023-12-06T03:16:54.322 INFO:teuthology.orchestra.run.smithi175.stdout:(10/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 14 MB/s | 368 kB 00:00 2023-12-06T03:16:54.355 INFO:teuthology.orchestra.run.smithi175.stdout:(11/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 3.4 MB/s | 114 kB 00:00 2023-12-06T03:16:54.506 INFO:teuthology.orchestra.run.smithi175.stdout:(12/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 19 MB/s | 13 MB 00:00 2023-12-06T03:16:54.531 INFO:teuthology.orchestra.run.smithi175.stdout:(13/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 828 kB/s | 145 kB 00:00 2023-12-06T03:16:54.581 INFO:teuthology.orchestra.run.smithi175.stdout:(14/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-12-06T03:16:54.615 INFO:teuthology.orchestra.run.smithi175.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 1.3 MB/s | 44 kB 00:00 2023-12-06T03:16:54.640 INFO:teuthology.orchestra.run.smithi175.stdout:(16/27): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-12-06T03:16:54.874 INFO:teuthology.orchestra.run.smithi175.stdout:(17/27): ceph-common-16.2.4-0.el8.x86_64.rpm 24 MB/s | 24 MB 00:01 2023-12-06T03:16:54.924 INFO:teuthology.orchestra.run.smithi175.stdout:(18/27): daxctl-libs-71.1-7.el8.x86_64.rpm 136 kB/s | 42 kB 00:00 2023-12-06T03:16:54.941 INFO:teuthology.orchestra.run.smithi175.stdout:(19/27): librabbitmq-0.9.0-4.el8.x86_64.rpm 156 kB/s | 47 kB 00:00 2023-12-06T03:16:54.992 INFO:teuthology.orchestra.run.smithi175.stdout:(20/27): ndctl-libs-71.1-7.el8.x86_64.rpm 685 kB/s | 80 kB 00:00 2023-12-06T03:16:55.258 INFO:teuthology.orchestra.run.smithi175.stdout:(21/27): fmt-6.2.1-1.el8.x86_64.rpm 349 kB/s | 116 kB 00:00 2023-12-06T03:16:55.342 INFO:teuthology.orchestra.run.smithi175.stdout:(22/27): gperftools-libs-2.7-9.el8.x86_64.rpm 765 kB/s | 306 kB 00:00 2023-12-06T03:16:55.367 INFO:teuthology.orchestra.run.smithi175.stdout:(23/27): leveldb-1.22-1.el8.x86_64.rpm 494 kB/s | 181 kB 00:00 2023-12-06T03:16:55.417 INFO:teuthology.orchestra.run.smithi175.stdout:(24/27): liboath-2.6.2-3.el8.x86_64.rpm 371 kB/s | 59 kB 00:00 2023-12-06T03:16:55.434 INFO:teuthology.orchestra.run.smithi175.stdout:(25/27): libunwind-1.3.1-3.el8.x86_64.rpm 815 kB/s | 75 kB 00:00 2023-12-06T03:16:55.518 INFO:teuthology.orchestra.run.smithi175.stdout:(26/27): librados2-16.2.4-0.el8.x86_64.rpm 37 MB/s | 3.7 MB 00:00 2023-12-06T03:16:55.577 INFO:teuthology.orchestra.run.smithi175.stdout:(27/27): librbd1-16.2.4-0.el8.x86_64.rpm 26 MB/s | 4.0 MB 00:00 2023-12-06T03:16:55.581 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:16:55.581 INFO:teuthology.orchestra.run.smithi175.stdout:Total 26 MB/s | 59 MB 00:02 2023-12-06T03:16:55.648 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:16:55.696 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:16:55.696 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:16:56.036 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:16:56.040 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:16:56.345 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:16:56.348 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:56.348 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:16:56.348 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:56.348 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:16:56.348 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: daxctl-libs x86_64 71.1-7.el8 baseos 42 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-06T03:16:56.349 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj x86_64 1.6.1-1.el8 appstream 145 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: ndctl-libs x86_64 71.1-7.el8 baseos 80 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2023-12-06T03:16:56.350 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout:Install 25 Packages 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 2 Packages 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 59 M 2023-12-06T03:16:56.351 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:16:56.891 INFO:teuthology.orchestra.run.smithi055.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 37 MB/s | 6.0 MB 00:00 2023-12-06T03:16:56.916 INFO:teuthology.orchestra.run.smithi055.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2023-12-06T03:16:56.950 INFO:teuthology.orchestra.run.smithi055.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 24 MB/s | 801 kB 00:00 2023-12-06T03:16:56.984 INFO:teuthology.orchestra.run.smithi055.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 14 MB/s | 486 kB 00:00 2023-12-06T03:16:57.092 INFO:teuthology.orchestra.run.smithi055.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 38 MB/s | 4.1 MB 00:00 2023-12-06T03:16:57.118 INFO:teuthology.orchestra.run.smithi055.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 1.7 MB/s | 44 kB 00:00 2023-12-06T03:16:57.143 INFO:teuthology.orchestra.run.smithi055.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 3.1 MB/s | 80 kB 00:00 2023-12-06T03:16:57.168 INFO:teuthology.orchestra.run.smithi055.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 8.4 MB/s | 213 kB 00:00 2023-12-06T03:16:57.202 INFO:teuthology.orchestra.run.smithi055.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 11 MB/s | 387 kB 00:00 2023-12-06T03:16:57.236 INFO:teuthology.orchestra.run.smithi055.stdout:(10/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 11 MB/s | 368 kB 00:00 2023-12-06T03:16:57.261 INFO:teuthology.orchestra.run.smithi055.stdout:(11/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 4.5 MB/s | 114 kB 00:00 2023-12-06T03:16:57.299 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:16:57.420 INFO:teuthology.orchestra.run.smithi055.stdout:(12/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 18 MB/s | 13 MB 00:00 2023-12-06T03:16:57.445 INFO:teuthology.orchestra.run.smithi055.stdout:(13/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 792 kB/s | 145 kB 00:00 2023-12-06T03:16:57.495 INFO:teuthology.orchestra.run.smithi055.stdout:(14/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-12-06T03:16:57.520 INFO:teuthology.orchestra.run.smithi055.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2023-12-06T03:16:57.526 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2023-12-06T03:16:57.546 INFO:teuthology.orchestra.run.smithi055.stdout:(16/27): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-12-06T03:16:57.738 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2023-12-06T03:16:57.838 INFO:teuthology.orchestra.run.smithi055.stdout:(17/27): ceph-common-16.2.4-0.el8.x86_64.rpm 22 MB/s | 24 MB 00:01 2023-12-06T03:16:57.855 INFO:teuthology.orchestra.run.smithi055.stdout:(18/27): daxctl-libs-71.1-7.el8.x86_64.rpm 126 kB/s | 42 kB 00:00 2023-12-06T03:16:57.880 INFO:teuthology.orchestra.run.smithi055.stdout:(19/27): librabbitmq-0.9.0-4.el8.x86_64.rpm 141 kB/s | 47 kB 00:00 2023-12-06T03:16:57.905 INFO:teuthology.orchestra.run.smithi055.stdout:(20/27): ndctl-libs-71.1-7.el8.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2023-12-06T03:16:57.907 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2023-12-06T03:16:58.139 INFO:teuthology.orchestra.run.smithi055.stdout:(21/27): fmt-6.2.1-1.el8.x86_64.rpm 410 kB/s | 116 kB 00:00 2023-12-06T03:16:58.164 INFO:teuthology.orchestra.run.smithi055.stdout:(22/27): leveldb-1.22-1.el8.x86_64.rpm 700 kB/s | 181 kB 00:00 2023-12-06T03:16:58.222 INFO:teuthology.orchestra.run.smithi055.stdout:(23/27): liboath-2.6.2-3.el8.x86_64.rpm 705 kB/s | 59 kB 00:00 2023-12-06T03:16:58.248 INFO:teuthology.orchestra.run.smithi055.stdout:(24/27): gperftools-libs-2.7-9.el8.x86_64.rpm 833 kB/s | 306 kB 00:00 2023-12-06T03:16:58.273 INFO:teuthology.orchestra.run.smithi055.stdout:(25/27): libunwind-1.3.1-3.el8.x86_64.rpm 690 kB/s | 75 kB 00:00 2023-12-06T03:16:58.330 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2023-12-06T03:16:58.365 INFO:teuthology.orchestra.run.smithi055.stdout:(26/27): librados2-16.2.4-0.el8.x86_64.rpm 26 MB/s | 3.7 MB 00:00 2023-12-06T03:16:58.432 INFO:teuthology.orchestra.run.smithi055.stdout:(27/27): librbd1-16.2.4-0.el8.x86_64.rpm 22 MB/s | 4.0 MB 00:00 2023-12-06T03:16:58.438 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:16:58.438 INFO:teuthology.orchestra.run.smithi055.stdout:Total 28 MB/s | 59 MB 00:02 2023-12-06T03:16:58.509 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:16:58.532 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2023-12-06T03:16:58.565 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:16:58.565 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:16:58.626 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2023-12-06T03:16:58.831 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2023-12-06T03:16:58.946 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:16:58.950 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:16:58.990 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2023-12-06T03:16:59.039 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2023-12-06T03:16:59.221 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/29 2023-12-06T03:16:59.407 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/29 2023-12-06T03:16:59.821 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/29 2023-12-06T03:16:59.965 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2023-12-06T03:17:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2023-12-06T03:17:00.163 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:17:00.229 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2023-12-06T03:17:00.372 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2023-12-06T03:17:00.462 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : daxctl-libs-71.1-7.el8.x86_64 11/29 2023-12-06T03:17:00.529 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: daxctl-libs-71.1-7.el8.x86_64 11/29 2023-12-06T03:17:00.687 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2023-12-06T03:17:00.858 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2023-12-06T03:17:00.873 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2023-12-06T03:17:01.032 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ndctl-libs-71.1-7.el8.x86_64 14/29 2023-12-06T03:17:01.058 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2023-12-06T03:17:01.091 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ndctl-libs-71.1-7.el8.x86_64 14/29 2023-12-06T03:17:01.232 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2023-12-06T03:17:01.501 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2023-12-06T03:17:01.675 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2023-12-06T03:17:01.697 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2023-12-06T03:17:01.764 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2023-12-06T03:17:01.996 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2023-12-06T03:17:02.031 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2023-12-06T03:17:02.128 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2023-12-06T03:17:02.183 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2023-12-06T03:17:02.238 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2023-12-06T03:17:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2023-12-06T03:17:02.412 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/29 2023-12-06T03:17:02.534 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2023-12-06T03:17:02.599 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/29 2023-12-06T03:17:02.665 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2023-12-06T03:17:02.734 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2023-12-06T03:17:02.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2023-12-06T03:17:03.066 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/29 2023-12-06T03:17:03.107 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2023-12-06T03:17:03.200 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2023-12-06T03:17:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2023-12-06T03:17:03.316 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2023-12-06T03:17:03.523 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2023-12-06T03:17:03.606 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2023-12-06T03:17:03.827 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : daxctl-libs-71.1-7.el8.x86_64 11/29 2023-12-06T03:17:03.895 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: daxctl-libs-71.1-7.el8.x86_64 11/29 2023-12-06T03:17:04.066 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2023-12-06T03:17:04.252 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2023-12-06T03:17:04.399 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ndctl-libs-71.1-7.el8.x86_64 14/29 2023-12-06T03:17:04.489 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ndctl-libs-71.1-7.el8.x86_64 14/29 2023-12-06T03:17:04.616 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2023-12-06T03:17:05.090 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2023-12-06T03:17:05.424 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2023-12-06T03:17:05.532 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2023-12-06T03:17:05.730 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2023-12-06T03:17:05.939 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2023-12-06T03:17:06.095 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2023-12-06T03:17:06.115 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2023-12-06T03:17:06.190 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2023-12-06T03:17:06.323 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2023-12-06T03:17:06.372 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2023-12-06T03:17:06.551 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2023-12-06T03:17:06.705 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2023-12-06T03:17:06.916 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2023-12-06T03:17:07.037 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2023-12-06T03:17:07.626 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2023-12-06T03:17:07.758 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2023-12-06T03:17:07.897 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2023-12-06T03:17:09.642 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2023-12-06T03:17:09.841 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2023-12-06T03:17:10.617 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2023-12-06T03:17:10.941 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2023-12-06T03:17:11.061 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2023-12-06T03:17:11.473 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /dev 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /run 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2023-12-06T03:17:26.340 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2023-12-06T03:17:26.341 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2023-12-06T03:17:26.341 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /dev 2023-12-06T03:17:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:27.672 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2023-12-06T03:17:27.737 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2023-12-06T03:17:27.737 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2023-12-06T03:17:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2023-12-06T03:17:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2023-12-06T03:17:28.624 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 15/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : daxctl-libs-71.1-7.el8.x86_64 18/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 19/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ndctl-libs-71.1-7.el8.x86_64 20/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2023-12-06T03:17:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2023-12-06T03:17:28.626 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2023-12-06T03:17:28.626 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2023-12-06T03:17:28.626 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2023-12-06T03:17:28.626 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2023-12-06T03:17:28.626 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2023-12-06T03:17:29.907 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2023-12-06T03:17:29.907 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:29.907 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2023-12-06T03:17:29.907 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.907 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: daxctl-libs-71.1-7.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: fmt-6.2.1-1.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: leveldb-1.22-1.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.6.1-1.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-06T03:17:29.908 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: ndctl-libs-71.1-7.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:29.909 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:17:30.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-test 2023-12-06T03:17:30.979 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:42 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:17:32.660 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:17:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:17:32.661 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:17:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:17:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:17:32.661 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2023-12-06T03:17:32.661 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2023-12-06T03:17:32.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 53 M 2023-12-06T03:17:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 229 M 2023-12-06T03:17:32.663 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /dev 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /run 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2023-12-06T03:17:32.725 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /dev 2023-12-06T03:17:32.726 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:32.910 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): libcephsqlite-16.2.4-0.el8.x86_64.rpm 2.8 MB/s | 169 kB 00:00 2023-12-06T03:17:32.936 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 2.3 MB/s | 203 kB 00:00 2023-12-06T03:17:32.986 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-12-06T03:17:33.011 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-12-06T03:17:33.037 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2023-12-06T03:17:33.604 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): ceph-test-16.2.4-0.el8.x86_64.rpm 69 MB/s | 52 MB 00:00 2023-12-06T03:17:33.606 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:17:33.606 INFO:teuthology.orchestra.run.smithi175.stdout:Total 56 MB/s | 53 MB 00:00 2023-12-06T03:17:33.649 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:17:33.670 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:17:33.670 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:17:33.980 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:17:33.983 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:17:34.146 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2023-12-06T03:17:34.214 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2023-12-06T03:17:34.214 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2023-12-06T03:17:34.457 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2023-12-06T03:17:34.457 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2023-12-06T03:17:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:17:34.737 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-12-06T03:17:34.919 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-12-06T03:17:35.122 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-12-06T03:17:35.191 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2023-12-06T03:17:35.219 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 15/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : daxctl-libs-71.1-7.el8.x86_64 18/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 19/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ndctl-libs-71.1-7.el8.x86_64 20/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2023-12-06T03:17:35.220 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2023-12-06T03:17:35.221 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2023-12-06T03:17:35.221 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2023-12-06T03:17:35.221 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2023-12-06T03:17:35.221 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2023-12-06T03:17:35.221 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2023-12-06T03:17:35.221 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2023-12-06T03:17:35.357 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2023-12-06T03:17:35.492 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2023-12-06T03:17:36.451 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2023-12-06T03:17:36.451 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:36.451 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2023-12-06T03:17:36.451 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.451 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: daxctl-libs-71.1-7.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: fmt-6.2.1-1.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: leveldb-1.22-1.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj-1.6.1-1.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.452 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: ndctl-libs-71.1-7.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:36.453 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:17:36.823 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-test 2023-12-06T03:17:37.797 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:46 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:17:39.666 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:17:39.667 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:17:39.667 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:17:39.667 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:17:39.667 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout:Install 6 Packages 2023-12-06T03:17:39.668 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:39.669 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 53 M 2023-12-06T03:17:39.669 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 229 M 2023-12-06T03:17:39.669 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:17:39.797 INFO:teuthology.orchestra.run.smithi055.stdout:(1/6): libcephsqlite-16.2.4-0.el8.x86_64.rpm 1.9 MB/s | 169 kB 00:00 2023-12-06T03:17:39.829 INFO:teuthology.orchestra.run.smithi055.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 1.7 MB/s | 203 kB 00:00 2023-12-06T03:17:39.879 INFO:teuthology.orchestra.run.smithi055.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-12-06T03:17:39.896 INFO:teuthology.orchestra.run.smithi055.stdout:(4/6): oniguruma-6.8.2-2.el8.x86_64.rpm 1.9 MB/s | 187 kB 00:00 2023-12-06T03:17:39.921 INFO:teuthology.orchestra.run.smithi055.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2023-12-06T03:17:40.357 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2023-12-06T03:17:40.597 INFO:teuthology.orchestra.run.smithi055.stdout:(6/6): ceph-test-16.2.4-0.el8.x86_64.rpm 59 MB/s | 52 MB 00:00 2023-12-06T03:17:40.599 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:17:40.599 INFO:teuthology.orchestra.run.smithi055.stdout:Total 57 MB/s | 53 MB 00:00 2023-12-06T03:17:40.606 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 6/6 2023-12-06T03:17:40.646 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:17:40.669 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:17:40.670 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:17:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 6/6 2023-12-06T03:17:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/6 2023-12-06T03:17:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/6 2023-12-06T03:17:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2023-12-06T03:17:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-12-06T03:17:40.881 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-12-06T03:17:41.022 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:17:41.025 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:17:41.533 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:17:41.649 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-12-06T03:17:41.649 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:41.650 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:17:41.650 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.6-8.el8.x86_64 2023-12-06T03:17:41.650 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 oniguruma-6.8.2-2.el8.x86_64 2023-12-06T03:17:41.650 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2023-12-06T03:17:41.650 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:41.650 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:17:41.780 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-12-06T03:17:41.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph 2023-12-06T03:17:41.983 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-12-06T03:17:42.319 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-12-06T03:17:42.393 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-12-06T03:17:42.579 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2023-12-06T03:17:42.712 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2023-12-06T03:17:42.720 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:54 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:17:44.397 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:17:44.401 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:17:44.401 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:17:44.401 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:17:44.401 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-12-06T03:17:44.402 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-06T03:17:44.403 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout:Install 32 Packages 2023-12-06T03:17:44.404 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:44.405 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 31 M 2023-12-06T03:17:44.405 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 111 M 2023-12-06T03:17:44.405 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:17:44.737 INFO:teuthology.orchestra.run.smithi175.stdout:(1/32): ceph-16.2.4-0.el8.x86_64.rpm 153 kB/s | 6.3 kB 00:00 2023-12-06T03:17:44.787 INFO:teuthology.orchestra.run.smithi175.stdout:(2/32): ceph-mgr-16.2.4-0.el8.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2023-12-06T03:17:44.846 INFO:teuthology.orchestra.run.smithi175.stdout:(3/32): ceph-mds-16.2.4-0.el8.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2023-12-06T03:17:44.880 INFO:teuthology.orchestra.run.smithi175.stdout:(4/32): ceph-mgr-modules-core-16.2.4-0.el8.noar 6.5 MB/s | 220 kB 00:00 2023-12-06T03:17:44.980 INFO:teuthology.orchestra.run.smithi175.stdout:(5/32): ceph-mon-16.2.4-0.el8.x86_64.rpm 21 MB/s | 5.0 MB 00:00 2023-12-06T03:17:45.005 INFO:teuthology.orchestra.run.smithi175.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-12-06T03:17:45.039 INFO:teuthology.orchestra.run.smithi175.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 3.0 MB/s | 103 kB 00:00 2023-12-06T03:17:45.064 INFO:teuthology.orchestra.run.smithi175.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 463 kB/s | 39 kB 00:00 2023-12-06T03:17:45.089 INFO:teuthology.orchestra.run.smithi175.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 1.1 MB/s | 57 kB 00:00 2023-12-06T03:17:45.140 INFO:teuthology.orchestra.run.smithi175.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 5.9 MB/s | 457 kB 00:00 2023-12-06T03:17:45.173 INFO:teuthology.orchestra.run.smithi175.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.9 MB/s | 249 kB 00:00 2023-12-06T03:17:45.357 INFO:teuthology.orchestra.run.smithi175.stdout:(12/32): ceph-osd-16.2.4-0.el8.x86_64.rpm 32 MB/s | 18 MB 00:00 2023-12-06T03:17:45.382 INFO:teuthology.orchestra.run.smithi175.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 523 kB/s | 109 kB 00:00 2023-12-06T03:17:45.441 INFO:teuthology.orchestra.run.smithi175.stdout:(14/32): python3-cryptography-3.2.1-6.el8.x86_6 2.1 MB/s | 641 kB 00:00 2023-12-06T03:17:45.566 INFO:teuthology.orchestra.run.smithi175.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 209 kB/s | 44 kB 00:00 2023-12-06T03:17:45.600 INFO:teuthology.orchestra.run.smithi175.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 854 kB/s | 185 kB 00:00 2023-12-06T03:17:45.633 INFO:teuthology.orchestra.run.smithi175.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 898 kB/s | 173 kB 00:00 2023-12-06T03:17:45.650 INFO:teuthology.orchestra.run.smithi175.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 220 kB/s | 11 kB 00:00 2023-12-06T03:17:45.684 INFO:teuthology.orchestra.run.smithi175.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 351 kB/s | 18 kB 00:00 2023-12-06T03:17:45.701 INFO:teuthology.orchestra.run.smithi175.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 966 kB/s | 49 kB 00:00 2023-12-06T03:17:45.743 INFO:teuthology.orchestra.run.smithi175.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.1 MB/s | 384 kB 00:00 2023-12-06T03:17:45.760 INFO:teuthology.orchestra.run.smithi175.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 773 kB/s | 59 kB 00:00 2023-12-06T03:17:45.794 INFO:teuthology.orchestra.run.smithi175.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2023-12-06T03:17:45.810 INFO:teuthology.orchestra.run.smithi175.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 232 kB/s | 16 kB 00:00 2023-12-06T03:17:45.827 INFO:teuthology.orchestra.run.smithi175.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 279 kB/s | 19 kB 00:00 2023-12-06T03:17:45.844 INFO:teuthology.orchestra.run.smithi175.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 474 kB/s | 24 kB 00:00 2023-12-06T03:17:45.861 INFO:teuthology.orchestra.run.smithi175.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 579 kB/s | 29 kB 00:00 2023-12-06T03:17:45.911 INFO:teuthology.orchestra.run.smithi175.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 329 kB/s | 27 kB 00:00 2023-12-06T03:17:45.937 INFO:teuthology.orchestra.run.smithi175.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.7 MB/s | 255 kB 00:00 2023-12-06T03:17:45.962 INFO:teuthology.orchestra.run.smithi175.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.4 MB/s | 251 kB 00:00 2023-12-06T03:17:45.987 INFO:teuthology.orchestra.run.smithi175.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2023-12-06T03:17:46.021 INFO:teuthology.orchestra.run.smithi175.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 391 kB/s | 23 kB 00:00 2023-12-06T03:17:46.022 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:17:46.022 INFO:teuthology.orchestra.run.smithi175.stdout:Total 19 MB/s | 31 MB 00:01 2023-12-06T03:17:46.102 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:17:46.140 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:17:46.140 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:17:46.457 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:17:46.460 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:17:47.575 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:17:47.769 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-12-06T03:17:48.005 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-12-06T03:17:48.096 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2023-12-06T03:17:48.238 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-12-06T03:17:48.360 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 6/6 2023-12-06T03:17:48.471 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-12-06T03:17:48.627 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-12-06T03:17:48.676 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 6/6 2023-12-06T03:17:48.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/6 2023-12-06T03:17:48.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/6 2023-12-06T03:17:48.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2023-12-06T03:17:48.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-12-06T03:17:48.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-12-06T03:17:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-12-06T03:17:48.992 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-12-06T03:17:49.161 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-12-06T03:17:49.351 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-12-06T03:17:49.534 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-12-06T03:17:49.534 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:49.534 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:17:49.535 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.6-8.el8.x86_64 2023-12-06T03:17:49.535 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 oniguruma-6.8.2-2.el8.x86_64 2023-12-06T03:17:49.535 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2023-12-06T03:17:49.535 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:49.535 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:17:49.581 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-12-06T03:17:49.769 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph 2023-12-06T03:17:49.777 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-12-06T03:17:50.018 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-12-06T03:17:50.367 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-12-06T03:17:50.617 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-12-06T03:17:50.758 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:59 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:17:50.772 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-12-06T03:17:51.022 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-12-06T03:17:51.190 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-12-06T03:17:51.501 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-12-06T03:17:51.711 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-12-06T03:17:51.909 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2023-12-06T03:17:52.116 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-12-06T03:17:52.341 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-12-06T03:17:52.566 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-12-06T03:17:52.647 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:17:52.650 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-12-06T03:17:52.651 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-12-06T03:17:52.652 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout:Install 32 Packages 2023-12-06T03:17:52.653 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:17:52.654 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 31 M 2023-12-06T03:17:52.654 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 111 M 2023-12-06T03:17:52.654 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:17:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-12-06T03:17:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-12-06T03:17:53.411 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-12-06T03:17:53.440 INFO:teuthology.orchestra.run.smithi055.stdout:(1/32): ceph-16.2.4-0.el8.x86_64.rpm 111 kB/s | 6.3 kB 00:00 2023-12-06T03:17:53.499 INFO:teuthology.orchestra.run.smithi055.stdout:(2/32): ceph-mgr-16.2.4-0.el8.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2023-12-06T03:17:53.558 INFO:teuthology.orchestra.run.smithi055.stdout:(3/32): ceph-mds-16.2.4-0.el8.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2023-12-06T03:17:53.583 INFO:teuthology.orchestra.run.smithi055.stdout:(4/32): ceph-mgr-modules-core-16.2.4-0.el8.noar 8.7 MB/s | 220 kB 00:00 2023-12-06T03:17:53.675 INFO:teuthology.orchestra.run.smithi055.stdout:(5/32): ceph-mon-16.2.4-0.el8.x86_64.rpm 21 MB/s | 5.0 MB 00:00 2023-12-06T03:17:53.732 INFO:teuthology.orchestra.run.smithi055.stdout:(6/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-12-06T03:17:53.792 INFO:teuthology.orchestra.run.smithi055.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.7 MB/s | 103 kB 00:00 2023-12-06T03:17:53.796 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 27/32 2023-12-06T03:17:53.809 INFO:teuthology.orchestra.run.smithi055.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 290 kB/s | 39 kB 00:00 2023-12-06T03:17:53.826 INFO:teuthology.orchestra.run.smithi055.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 1.7 MB/s | 57 kB 00:00 2023-12-06T03:17:53.876 INFO:teuthology.orchestra.run.smithi055.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 6.7 MB/s | 457 kB 00:00 2023-12-06T03:17:53.927 INFO:teuthology.orchestra.run.smithi055.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.4 MB/s | 249 kB 00:00 2023-12-06T03:17:53.977 INFO:teuthology.orchestra.run.smithi055.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 2.1 MB/s | 109 kB 00:00 2023-12-06T03:17:53.992 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2023-12-06T03:17:54.327 INFO:teuthology.orchestra.run.smithi055.stdout:(13/32): ceph-osd-16.2.4-0.el8.x86_64.rpm 22 MB/s | 18 MB 00:00 2023-12-06T03:17:54.361 INFO:teuthology.orchestra.run.smithi055.stdout:(14/32): python3-cryptography-3.2.1-6.el8.x86_6 1.3 MB/s | 641 kB 00:00 2023-12-06T03:17:54.558 INFO:teuthology.orchestra.run.smithi055.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 75 kB/s | 44 kB 00:00 2023-12-06T03:17:54.620 INFO:teuthology.orchestra.run.smithi055.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 646 kB/s | 185 kB 00:00 2023-12-06T03:17:54.645 INFO:teuthology.orchestra.run.smithi055.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 608 kB/s | 173 kB 00:00 2023-12-06T03:17:54.679 INFO:teuthology.orchestra.run.smithi055.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-12-06T03:17:54.704 INFO:teuthology.orchestra.run.smithi055.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 301 kB/s | 18 kB 00:00 2023-12-06T03:17:54.738 INFO:teuthology.orchestra.run.smithi055.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 832 kB/s | 49 kB 00:00 2023-12-06T03:17:54.763 INFO:teuthology.orchestra.run.smithi055.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 1.0 MB/s | 59 kB 00:00 2023-12-06T03:17:54.788 INFO:teuthology.orchestra.run.smithi055.stdout:(22/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.6 MB/s | 384 kB 00:00 2023-12-06T03:17:54.822 INFO:teuthology.orchestra.run.smithi055.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2023-12-06T03:17:54.847 INFO:teuthology.orchestra.run.smithi055.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 317 kB/s | 19 kB 00:00 2023-12-06T03:17:54.881 INFO:teuthology.orchestra.run.smithi055.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 409 kB/s | 24 kB 00:00 2023-12-06T03:17:54.906 INFO:teuthology.orchestra.run.smithi055.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 499 kB/s | 29 kB 00:00 2023-12-06T03:17:54.957 INFO:teuthology.orchestra.run.smithi055.stdout:(27/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-12-06T03:17:54.982 INFO:teuthology.orchestra.run.smithi055.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 274 kB/s | 27 kB 00:00 2023-12-06T03:17:55.057 INFO:teuthology.orchestra.run.smithi055.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.7 MB/s | 255 kB 00:00 2023-12-06T03:17:55.082 INFO:teuthology.orchestra.run.smithi055.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 851 kB/s | 85 kB 00:00 2023-12-06T03:17:55.107 INFO:teuthology.orchestra.run.smithi055.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.6 MB/s | 251 kB 00:00 2023-12-06T03:17:55.133 INFO:teuthology.orchestra.run.smithi055.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 305 kB/s | 23 kB 00:00 2023-12-06T03:17:55.137 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:17:55.138 INFO:teuthology.orchestra.run.smithi055.stdout:Total 12 MB/s | 31 MB 00:02 2023-12-06T03:17:55.228 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:17:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:17:55.272 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:17:55.519 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2023-12-06T03:17:55.659 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:17:55.663 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:17:55.675 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2023-12-06T03:17:56.107 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2023-12-06T03:17:56.326 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2023-12-06T03:17:56.548 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2023-12-06T03:17:56.713 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2023-12-06T03:17:56.731 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2023-12-06T03:17:56.879 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 32/32 2023-12-06T03:17:56.881 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:17:57.075 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-12-06T03:17:57.283 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 32/32 2023-12-06T03:17:57.283 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/32 2023-12-06T03:17:57.283 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/32 2023-12-06T03:17:57.283 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/32 2023-12-06T03:17:57.283 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-12-06T03:17:57.284 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-12-06T03:17:57.285 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-12-06T03:17:57.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-12-06T03:17:57.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-12-06T03:17:57.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-12-06T03:17:57.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-12-06T03:17:57.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-12-06T03:17:57.286 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-12-06T03:17:57.303 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-12-06T03:17:57.539 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-12-06T03:17:57.767 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2023-12-06T03:17:57.931 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-12-06T03:17:58.115 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-12-06T03:17:58.263 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-12-06T03:17:58.407 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-12-06T03:17:58.450 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-12-06T03:17:58.451 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:58.451 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:17:58.451 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:58.451 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:58.451 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-06T03:17:58.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-06T03:17:58.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-06T03:17:58.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-06T03:17:58.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-06T03:17:58.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-06T03:17:58.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-06T03:17:58.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-2.6-1.el8.noarch 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-06T03:17:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-06T03:17:58.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-06T03:17:58.456 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-06T03:17:58.456 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-06T03:17:58.456 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:17:58.456 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:17:58.621 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-12-06T03:17:58.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base 2023-12-06T03:17:58.815 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-12-06T03:17:59.020 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-12-06T03:17:59.286 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-12-06T03:17:59.543 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:11 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:17:59.654 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2023-12-06T03:17:59.908 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 14/32 2023-12-06T03:18:00.094 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-12-06T03:18:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-12-06T03:18:00.537 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-12-06T03:18:00.817 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-12-06T03:18:00.998 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:18:01.010 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-12-06T03:18:01.214 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2023-12-06T03:18:01.221 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:01.222 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:18:01.222 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:01.405 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-12-06T03:18:01.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install cephadm 2023-12-06T03:18:01.697 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-12-06T03:18:01.942 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-12-06T03:18:02.157 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-12-06T03:18:02.441 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:14 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:02.560 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-12-06T03:18:02.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-12-06T03:18:03.329 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 27/32 2023-12-06T03:18:03.559 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2023-12-06T03:18:04.121 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 71 k 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 290 k 2023-12-06T03:18:04.122 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:04.182 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm-16.2.4-0.el8.noarch.rpm 1.2 MB/s | 71 kB 00:00 2023-12-06T03:18:04.183 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:04.183 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.1 MB/s | 71 kB 00:00 2023-12-06T03:18:04.183 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:04.190 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:04.190 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:04.192 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:04.195 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:04.238 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:04.724 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:04.927 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:05.154 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:05.249 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2023-12-06T03:18:05.467 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2023-12-06T03:18:05.881 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:05.881 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:05.881 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:05.882 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:16.2.4-0.el8.noarch 2023-12-06T03:18:05.882 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:05.882 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:05.948 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2023-12-06T03:18:06.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-immutable-object-cache 2023-12-06T03:18:06.159 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2023-12-06T03:18:06.406 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2023-12-06T03:18:06.625 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2023-12-06T03:18:06.649 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2023-12-06T03:18:06.776 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 32/32 2023-12-06T03:18:06.944 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:18 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:07.252 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 32/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-12-06T03:18:07.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 13/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-12-06T03:18:07.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-12-06T03:18:07.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-12-06T03:18:08.646 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:08.647 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:08.648 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:18:08.648 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:08.648 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 163 k 2023-12-06T03:18:08.648 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 587 k 2023-12-06T03:18:08.648 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:08.703 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 2.9 MB/s | 163 kB 00:00 2023-12-06T03:18:08.704 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:08.704 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.9 MB/s | 163 kB 00:00 2023-12-06T03:18:08.704 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:08.714 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:08.714 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:08.762 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:08.765 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:08.876 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:09.044 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:09.046 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-2.6-1.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-06T03:18:09.047 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:09.048 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:09.216 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-base 2023-12-06T03:18:09.468 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:10.136 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:10.136 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:10.136 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:10.136 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:10.136 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:10.136 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:10.211 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:19 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:10.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr 2023-12-06T03:18:11.234 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:23 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:11.839 INFO:teuthology.orchestra.run.smithi055.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:18:12.088 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:12.089 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:18:12.089 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:12.211 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install cephadm 2023-12-06T03:18:12.668 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:18:12.891 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:12.893 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:18:12.893 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:13.202 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:22 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:13.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-dashboard 2023-12-06T03:18:14.225 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:26 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:18:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 71 k 2023-12-06T03:18:15.069 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 290 k 2023-12-06T03:18:15.070 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:18:15.106 INFO:teuthology.orchestra.run.smithi055.stdout:cephadm-16.2.4-0.el8.noarch.rpm 1.9 MB/s | 71 kB 00:00 2023-12-06T03:18:15.106 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:15.106 INFO:teuthology.orchestra.run.smithi055.stdout:Total 1.9 MB/s | 71 kB 00:00 2023-12-06T03:18:15.107 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:18:15.114 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:18:15.115 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:18:15.117 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:18:15.120 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:18:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:18:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:15.900 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:15.901 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:15.901 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:18:15.901 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:15.901 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2023-12-06T03:18:15.902 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:15.903 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 3.0 M 2023-12-06T03:18:15.903 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 15 M 2023-12-06T03:18:15.903 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:15.963 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:16.238 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:16.355 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 600 kB/s | 24 kB 00:00 2023-12-06T03:18:16.380 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 137 kB/s | 8.9 kB 00:00 2023-12-06T03:18:16.439 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 21 MB/s | 2.7 MB 00:00 2023-12-06T03:18:16.464 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 390 kB/s | 43 kB 00:00 2023-12-06T03:18:16.664 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 118 kB/s | 34 kB 00:00 2023-12-06T03:18:16.906 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 421 kB/s | 196 kB 00:00 2023-12-06T03:18:16.908 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:16.908 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.9 MB/s | 3.0 MB 00:01 2023-12-06T03:18:16.919 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:16.929 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:16.929 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:17.020 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2023-12-06T03:18:17.020 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:17.020 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:18:17.020 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:16.2.4-0.el8.noarch 2023-12-06T03:18:17.020 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:17.020 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:17.110 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:17.113 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:17.276 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-immutable-object-cache 2023-12-06T03:18:17.386 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:17.623 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-12-06T03:18:17.851 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-12-06T03:18:18.027 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-12-06T03:18:18.181 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2023-12-06T03:18:18.275 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:27 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:19.150 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2023-12-06T03:18:19.412 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:18:19.695 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:18:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2023-12-06T03:18:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2023-12-06T03:18:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2023-12-06T03:18:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-12-06T03:18:19.696 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-12-06T03:18:20.134 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:20.134 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:20.134 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repo Size 2023-12-06T03:18:20.134 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:20.135 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 163 k 2023-12-06T03:18:20.136 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 587 k 2023-12-06T03:18:20.136 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:18:20.185 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 3.2 MB/s | 163 kB 00:00 2023-12-06T03:18:20.186 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:20.186 INFO:teuthology.orchestra.run.smithi055.stdout:Total 3.2 MB/s | 163 kB 00:00 2023-12-06T03:18:20.186 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:18:20.199 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:18:20.199 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:18:20.253 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:18:20.256 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:18:20.375 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:20.417 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:20.540 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:20.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-diskprediction-local 2023-12-06T03:18:21.010 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:21.392 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:33 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:21.756 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:21.757 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:21.757 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:18:21.757 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:21.757 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:21.757 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:21.977 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-mgr 2023-12-06T03:18:22.967 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:31 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:23.094 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:23.095 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================================ 2023-12-06T03:18:23.095 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:18:23.095 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================================ 2023-12-06T03:18:23.095 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================================ 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout:Install 8 Packages 2023-12-06T03:18:23.096 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:23.097 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 37 M 2023-12-06T03:18:23.097 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 210 M 2023-12-06T03:18:23.097 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:23.519 INFO:teuthology.orchestra.run.smithi175.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.4-0.e 46 MB/s | 8.6 MB 00:00 2023-12-06T03:18:23.669 INFO:teuthology.orchestra.run.smithi175.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 25 MB/s | 3.7 MB 00:00 2023-12-06T03:18:23.744 INFO:teuthology.orchestra.run.smithi175.stdout:(3/8): openblas-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2023-12-06T03:18:23.769 INFO:teuthology.orchestra.run.smithi175.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.2 MB/s | 225 kB 00:00 2023-12-06T03:18:23.836 INFO:teuthology.orchestra.run.smithi175.stdout:(5/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 9.6 MB/s | 4.9 MB 00:00 2023-12-06T03:18:23.928 INFO:teuthology.orchestra.run.smithi175.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 1.8 MB/s | 172 kB 00:00 2023-12-06T03:18:23.986 INFO:teuthology.orchestra.run.smithi175.stdout:(7/8): libgfortran-8.5.0-21.el8.x86_64.rpm 2.9 MB/s | 645 kB 00:00 2023-12-06T03:18:24.255 INFO:teuthology.orchestra.run.smithi175.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 27 MB/s | 14 MB 00:00 2023-12-06T03:18:24.255 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:24.255 INFO:teuthology.orchestra.run.smithi175.stdout:Total 32 MB/s | 37 MB 00:01 2023-12-06T03:18:24.487 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:24.505 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:24.506 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:24.585 INFO:teuthology.orchestra.run.smithi055.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:18:24.819 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:24.822 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:24.836 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:24.837 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:18:24.837 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:24.969 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-mgr-dashboard 2023-12-06T03:18:25.474 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:25.616 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2023-12-06T03:18:25.744 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2023-12-06T03:18:25.890 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2023-12-06T03:18:25.950 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:34 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:26.502 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2023-12-06T03:18:26.690 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-12-06T03:18:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-12-06T03:18:27.490 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-12-06T03:18:27.840 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:27.841 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:27.841 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:18:27.841 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:27.841 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:18:27.841 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout:Install 6 Packages 2023-12-06T03:18:27.842 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:27.843 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 3.0 M 2023-12-06T03:18:27.843 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 15 M 2023-12-06T03:18:27.843 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:18:28.146 INFO:teuthology.orchestra.run.smithi055.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 445 kB/s | 24 kB 00:00 2023-12-06T03:18:28.171 INFO:teuthology.orchestra.run.smithi055.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 113 kB/s | 8.9 kB 00:00 2023-12-06T03:18:28.188 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-12-06T03:18:28.238 INFO:teuthology.orchestra.run.smithi055.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 18 MB/s | 2.7 MB 00:00 2023-12-06T03:18:28.263 INFO:teuthology.orchestra.run.smithi055.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 364 kB/s | 43 kB 00:00 2023-12-06T03:18:28.480 INFO:teuthology.orchestra.run.smithi055.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 109 kB/s | 34 kB 00:00 2023-12-06T03:18:28.518 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-12-06T03:18:28.706 INFO:teuthology.orchestra.run.smithi055.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 421 kB/s | 196 kB 00:00 2023-12-06T03:18:28.708 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:28.708 INFO:teuthology.orchestra.run.smithi055.stdout:Total 3.4 MB/s | 3.0 MB 00:00 2023-12-06T03:18:28.720 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:18:28.732 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:18:28.732 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:18:28.748 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-12-06T03:18:28.918 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:18:28.922 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:18:29.219 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:18:29.424 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-12-06T03:18:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-12-06T03:18:29.882 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-12-06T03:18:30.038 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2023-12-06T03:18:30.515 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-12-06T03:18:31.090 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2023-12-06T03:18:31.347 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:18:31.676 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:18:31.676 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2023-12-06T03:18:31.676 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2023-12-06T03:18:31.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2023-12-06T03:18:31.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-12-06T03:18:31.677 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-12-06T03:18:31.809 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-12-06T03:18:32.042 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2023-12-06T03:18:32.379 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2023-12-06T03:18:32.379 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2023-12-06T03:18:32.379 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-12-06T03:18:32.379 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-12-06T03:18:32.380 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-12-06T03:18:32.380 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-12-06T03:18:32.380 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-12-06T03:18:32.380 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2023-12-06T03:18:32.444 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:32.579 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-mgr-diskprediction-local 2023-12-06T03:18:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2023-12-06T03:18:33.090 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.15-4.el8.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:33.091 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:33.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-rook 2023-12-06T03:18:33.560 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:42 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:34.056 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:46 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================================ 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================================ 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2023-12-06T03:18:35.453 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================================ 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout:Install 8 Packages 2023-12-06T03:18:35.454 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:35.455 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 37 M 2023-12-06T03:18:35.455 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 210 M 2023-12-06T03:18:35.455 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:18:35.743 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:35.744 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:35.744 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-06T03:18:35.745 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-12-06T03:18:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-12-06T03:18:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-12-06T03:18:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-12-06T03:18:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-06T03:18:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-06T03:18:35.746 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:35.747 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:35.747 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:35.747 INFO:teuthology.orchestra.run.smithi175.stdout:Install 13 Packages 2023-12-06T03:18:35.747 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:35.748 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 1.9 M 2023-12-06T03:18:35.748 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 24 M 2023-12-06T03:18:35.748 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:36.028 INFO:teuthology.orchestra.run.smithi175.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 756 kB/s | 27 kB 00:00 2023-12-06T03:18:36.054 INFO:teuthology.orchestra.run.smithi175.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 438 kB/s | 27 kB 00:00 2023-12-06T03:18:36.078 INFO:teuthology.orchestra.run.smithi175.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 232 kB/s | 20 kB 00:00 2023-12-06T03:18:36.129 INFO:teuthology.orchestra.run.smithi175.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-12-06T03:18:36.146 INFO:teuthology.orchestra.run.smithi175.stdout:(5/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-12-06T03:18:36.171 INFO:teuthology.orchestra.run.smithi175.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.6 MB/s | 155 kB 00:00 2023-12-06T03:18:36.196 INFO:teuthology.orchestra.run.smithi175.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 634 kB/s | 43 kB 00:00 2023-12-06T03:18:36.230 INFO:teuthology.orchestra.run.smithi055.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.4-0.e 16 MB/s | 8.6 MB 00:00 2023-12-06T03:18:36.230 INFO:teuthology.orchestra.run.smithi175.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 279 kB/s | 16 kB 00:00 2023-12-06T03:18:36.247 INFO:teuthology.orchestra.run.smithi175.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 327 kB/s | 33 kB 00:00 2023-12-06T03:18:36.296 INFO:teuthology.orchestra.run.smithi175.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 817 kB/s | 81 kB 00:00 2023-12-06T03:18:36.322 INFO:teuthology.orchestra.run.smithi175.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2023-12-06T03:18:36.348 INFO:teuthology.orchestra.run.smithi175.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 1.2 MB/s | 61 kB 00:00 2023-12-06T03:18:36.423 INFO:teuthology.orchestra.run.smithi175.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 6.1 MB/s | 1.2 MB 00:00 2023-12-06T03:18:36.423 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:36.423 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.8 MB/s | 1.9 MB 00:00 2023-12-06T03:18:36.475 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:36.490 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:36.490 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:36.577 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:36.579 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:36.705 INFO:teuthology.orchestra.run.smithi055.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 4.6 MB/s | 4.7 MB 00:01 2023-12-06T03:18:36.797 INFO:teuthology.orchestra.run.smithi055.stdout:(3/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 4.4 MB/s | 4.9 MB 00:01 2023-12-06T03:18:36.822 INFO:teuthology.orchestra.run.smithi055.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.9 MB/s | 225 kB 00:00 2023-12-06T03:18:36.906 INFO:teuthology.orchestra.run.smithi055.stdout:(5/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 5.5 MB/s | 3.7 MB 00:00 2023-12-06T03:18:37.079 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:37.173 INFO:teuthology.orchestra.run.smithi055.stdout:(6/8): python3-scipy-1.0.0-21.module_el8.5.0+77 36 MB/s | 14 MB 00:00 2023-12-06T03:18:37.198 INFO:teuthology.orchestra.run.smithi055.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 591 kB/s | 172 kB 00:00 2023-12-06T03:18:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-12-06T03:18:37.365 INFO:teuthology.orchestra.run.smithi055.stdout:(8/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.2 MB/s | 645 kB 00:00 2023-12-06T03:18:37.365 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:37.365 INFO:teuthology.orchestra.run.smithi055.stdout:Total 19 MB/s | 37 MB 00:01 2023-12-06T03:18:37.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-12-06T03:18:37.615 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:18:37.635 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:18:37.635 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:18:37.749 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2023-12-06T03:18:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-12-06T03:18:38.003 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:18:38.006 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:18:38.183 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-12-06T03:18:38.441 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-12-06T03:18:38.703 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:18:38.741 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-12-06T03:18:38.837 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2023-12-06T03:18:38.984 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-12-06T03:18:38.984 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2023-12-06T03:18:39.162 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2023-12-06T03:18:39.805 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2023-12-06T03:18:39.995 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-12-06T03:18:39.996 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-12-06T03:18:40.286 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-12-06T03:18:40.496 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-12-06T03:18:40.659 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-12-06T03:18:40.687 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-12-06T03:18:40.812 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-12-06T03:18:40.879 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2023-12-06T03:18:41.175 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2023-12-06T03:18:41.175 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2023-12-06T03:18:41.175 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-12-06T03:18:41.175 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-12-06T03:18:41.176 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2023-12-06T03:18:41.587 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-12-06T03:18:41.948 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-12-06T03:18:41.948 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:41.948 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:41.948 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2023-12-06T03:18:41.948 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-06T03:18:41.948 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:41.949 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:42.019 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-12-06T03:18:42.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-cephadm 2023-12-06T03:18:42.269 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-12-06T03:18:42.934 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:54 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:44.248 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-12-06T03:18:44.629 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:44.629 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:44.629 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2023-12-06T03:18:44.629 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:44.630 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2023-12-06T03:18:44.631 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:44.631 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 5.8 M 2023-12-06T03:18:44.631 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 24 M 2023-12-06T03:18:44.631 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:45.013 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 1.9 MB/s | 89 kB 00:00 2023-12-06T03:18:45.063 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.5 MB/s | 538 kB 00:00 2023-12-06T03:18:45.164 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-12-06T03:18:45.414 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-apipkg-1.5-6.el8.noarch.rpm 53 kB/s | 21 kB 00:00 2023-12-06T03:18:45.547 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 130 kB/s | 49 kB 00:00 2023-12-06T03:18:45.677 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-12-06T03:18:45.681 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 576 kB/s | 355 kB 00:00 2023-12-06T03:18:45.683 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:45.683 INFO:teuthology.orchestra.run.smithi175.stdout:Total 5.5 MB/s | 5.8 MB 00:01 2023-12-06T03:18:45.736 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:45.745 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:45.745 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:45.858 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:45.861 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:45.869 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2023-12-06T03:18:46.133 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:46.234 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-12-06T03:18:46.235 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2023-12-06T03:18:46.370 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2023-12-06T03:18:46.686 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2023-12-06T03:18:47.022 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-8.5.0-21.el8.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-8.5.0-21.el8.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.15-4.el8.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:47.023 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:47.141 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-mgr-rook 2023-12-06T03:18:47.608 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2023-12-06T03:18:47.959 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2023-12-06T03:18:48.130 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:01:57 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:48.208 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 5/6 2023-12-06T03:18:48.411 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:18:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:18:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2023-12-06T03:18:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2023-12-06T03:18:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/6 2023-12-06T03:18:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2023-12-06T03:18:48.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.5.1-7.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.10.1-3.el8.noarch python3-remoto-1.2.1-1.el8.noarch 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:49.399 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:49.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-fuse 2023-12-06T03:18:49.992 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-12-06T03:18:49.994 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout:Install 13 Packages 2023-12-06T03:18:49.995 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:49.996 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 1.9 M 2023-12-06T03:18:49.996 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 24 M 2023-12-06T03:18:49.996 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:18:50.406 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:02 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:50.450 INFO:teuthology.orchestra.run.smithi055.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 499 kB/s | 27 kB 00:00 2023-12-06T03:18:50.474 INFO:teuthology.orchestra.run.smithi055.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 336 kB/s | 27 kB 00:00 2023-12-06T03:18:50.498 INFO:teuthology.orchestra.run.smithi055.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 192 kB/s | 20 kB 00:00 2023-12-06T03:18:50.539 INFO:teuthology.orchestra.run.smithi055.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.6 MB/s | 110 kB 00:00 2023-12-06T03:18:50.565 INFO:teuthology.orchestra.run.smithi055.stdout:(5/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-12-06T03:18:50.598 INFO:teuthology.orchestra.run.smithi055.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.5 MB/s | 155 kB 00:00 2023-12-06T03:18:50.643 INFO:teuthology.orchestra.run.smithi055.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 413 kB/s | 43 kB 00:00 2023-12-06T03:18:50.707 INFO:teuthology.orchestra.run.smithi055.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 150 kB/s | 16 kB 00:00 2023-12-06T03:18:50.733 INFO:teuthology.orchestra.run.smithi055.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 197 kB/s | 33 kB 00:00 2023-12-06T03:18:50.766 INFO:teuthology.orchestra.run.smithi055.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 663 kB/s | 81 kB 00:00 2023-12-06T03:18:50.800 INFO:teuthology.orchestra.run.smithi055.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 959 kB/s | 64 kB 00:00 2023-12-06T03:18:50.825 INFO:teuthology.orchestra.run.smithi055.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 1.0 MB/s | 61 kB 00:00 2023-12-06T03:18:50.896 INFO:teuthology.orchestra.run.smithi055.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 6.3 MB/s | 1.2 MB 00:00 2023-12-06T03:18:50.896 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:50.896 INFO:teuthology.orchestra.run.smithi055.stdout:Total 2.1 MB/s | 1.9 MB 00:00 2023-12-06T03:18:50.955 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:18:50.975 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:18:50.975 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:18:51.071 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:18:51.076 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:18:51.617 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:18:51.862 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-12-06T03:18:52.084 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-12-06T03:18:52.105 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:52.105 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:52.105 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:Install 2 Packages 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:52.106 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 968 k 2023-12-06T03:18:52.107 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 3.2 M 2023-12-06T03:18:52.107 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:52.200 INFO:teuthology.orchestra.run.smithi175.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 16 MB/s | 885 kB 00:00 2023-12-06T03:18:52.225 INFO:teuthology.orchestra.run.smithi175.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2023-12-06T03:18:52.225 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:52.226 INFO:teuthology.orchestra.run.smithi175.stdout:Total 7.8 MB/s | 968 kB 00:00 2023-12-06T03:18:52.236 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:52.256 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:52.257 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:52.342 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2023-12-06T03:18:52.343 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:52.346 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:52.529 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:52.544 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-12-06T03:18:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-12-06T03:18:52.797 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-12-06T03:18:52.935 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2023-12-06T03:18:53.071 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-12-06T03:18:53.341 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2023-12-06T03:18:53.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2023-12-06T03:18:53.358 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-12-06T03:18:53.602 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-12-06T03:18:54.036 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-12-06T03:18:54.036 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:54.036 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:54.036 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-12-06T03:18:54.036 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:54.036 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:54.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados-devel 2023-12-06T03:18:54.773 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-12-06T03:18:55.069 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-12-06T03:18:55.115 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:07 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:55.260 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-12-06T03:18:55.478 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-12-06T03:18:55.668 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2023-12-06T03:18:56.034 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2023-12-06T03:18:56.034 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2023-12-06T03:18:56.034 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-12-06T03:18:56.034 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-12-06T03:18:56.035 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2023-12-06T03:18:56.785 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:18:56.786 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:56.786 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:18:56.786 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:56.786 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 131 k 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 439 k 2023-12-06T03:18:56.787 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:18:56.836 INFO:teuthology.orchestra.run.smithi175.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 2.6 MB/s | 131 kB 00:00 2023-12-06T03:18:56.837 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:18:56.837 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.5 MB/s | 131 kB 00:00 2023-12-06T03:18:56.837 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:18:56.842 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:18:56.843 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:18:56.894 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:18:56.898 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:18:56.901 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-12-06T03:18:56.901 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:56.901 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:18:56.901 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:56.902 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:18:57.035 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-mgr-cephadm 2023-12-06T03:18:57.049 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:18:57.250 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:57.539 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:58.030 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:07 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:18:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:18:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:58.208 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:18:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2023-12-06T03:18:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:18:58.208 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:18:58.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs2 2023-12-06T03:18:59.307 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:11 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:18:59.911 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:18:59.912 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:18:59.913 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:18:59.914 INFO:teuthology.orchestra.run.smithi055.stdout:Install 6 Packages 2023-12-06T03:18:59.914 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:18:59.914 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 5.8 M 2023-12-06T03:18:59.914 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 24 M 2023-12-06T03:18:59.914 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:00.133 INFO:teuthology.orchestra.run.smithi055.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 2.7 MB/s | 89 kB 00:00 2023-12-06T03:19:00.191 INFO:teuthology.orchestra.run.smithi055.stdout:(2/6): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.8 MB/s | 538 kB 00:00 2023-12-06T03:19:00.291 INFO:teuthology.orchestra.run.smithi055.stdout:(3/6): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-12-06T03:19:00.317 INFO:teuthology.orchestra.run.smithi055.stdout:(4/6): python3-apipkg-1.5-6.el8.noarch.rpm 115 kB/s | 21 kB 00:00 2023-12-06T03:19:00.442 INFO:teuthology.orchestra.run.smithi055.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 327 kB/s | 49 kB 00:00 2023-12-06T03:19:00.467 INFO:teuthology.orchestra.run.smithi055.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.3 MB/s | 355 kB 00:00 2023-12-06T03:19:00.470 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:00.470 INFO:teuthology.orchestra.run.smithi055.stdout:Total 10 MB/s | 5.8 MB 00:00 2023-12-06T03:19:00.528 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:00.540 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:00.540 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:00.659 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:00.748 INFO:teuthology.orchestra.run.smithi175.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:00.973 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:00.974 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:00.974 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:00.986 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:01.080 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs-devel 2023-12-06T03:19:01.283 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2023-12-06T03:19:01.629 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2023-12-06T03:19:01.939 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:13 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:02.640 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2023-12-06T03:19:03.099 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2023-12-06T03:19:03.334 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 5/6 2023-12-06T03:19:03.559 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:19:03.610 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:03.610 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:19:03.611 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:03.612 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 24 k 2023-12-06T03:19:03.612 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 94 k 2023-12-06T03:19:03.612 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:19:03.660 INFO:teuthology.orchestra.run.smithi175.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 502 kB/s | 24 kB 00:00 2023-12-06T03:19:03.661 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:03.661 INFO:teuthology.orchestra.run.smithi175.stdout:Total 489 kB/s | 24 kB 00:00 2023-12-06T03:19:03.662 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:19:03.665 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:19:03.665 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:19:03.667 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:19:03.670 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:19:03.722 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:19:03.883 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2023-12-06T03:19:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2023-12-06T03:19:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2023-12-06T03:19:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/6 2023-12-06T03:19:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2023-12-06T03:19:03.884 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2023-12-06T03:19:03.949 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:04.217 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:04.684 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2023-12-06T03:19:04.684 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:04.684 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:04.684 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2023-12-06T03:19:04.685 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.5.1-7.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2023-12-06T03:19:04.685 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.10.1-3.el8.noarch python3-remoto-1.2.1-1.el8.noarch 2023-12-06T03:19:04.685 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:04.685 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:04.843 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-fuse 2023-12-06T03:19:04.854 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:04.854 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:04.854 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:19:04.855 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:04.855 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:04.855 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:04.990 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados2 2023-12-06T03:19:05.831 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:14 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:05.843 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:17 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:07.309 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:07.534 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:07.536 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:07.536 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:07.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librbd1 2023-12-06T03:19:07.732 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-12-06T03:19:07.733 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout:Install 2 Packages 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 968 k 2023-12-06T03:19:07.734 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 3.2 M 2023-12-06T03:19:07.735 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:08.329 INFO:teuthology.orchestra.run.smithi055.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 13 MB/s | 885 kB 00:00 2023-12-06T03:19:08.354 INFO:teuthology.orchestra.run.smithi055.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 907 kB/s | 83 kB 00:00 2023-12-06T03:19:08.354 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:08.355 INFO:teuthology.orchestra.run.smithi055.stdout:Total 1.5 MB/s | 968 kB 00:00 2023-12-06T03:19:08.368 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:08.392 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:08.392 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:08.495 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:08.498 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:08.521 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:20 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:08.713 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:08.969 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2023-12-06T03:19:09.143 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2023-12-06T03:19:09.593 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2023-12-06T03:19:09.593 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2023-12-06T03:19:09.982 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:10.207 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:10.208 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:10.208 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:10.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rados 2023-12-06T03:19:10.361 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2023-12-06T03:19:10.362 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:10.362 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:10.362 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2023-12-06T03:19:10.362 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:10.362 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:10.635 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install librados-devel 2023-12-06T03:19:11.190 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:23 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:11.628 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:20 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:12.635 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:12.875 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:12.876 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:12.876 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:12.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rgw 2023-12-06T03:19:13.499 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:13.499 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:13.499 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:13.499 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:13.500 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 131 k 2023-12-06T03:19:13.501 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 439 k 2023-12-06T03:19:13.501 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:13.861 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:25 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:14.095 INFO:teuthology.orchestra.run.smithi055.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 221 kB/s | 131 kB 00:00 2023-12-06T03:19:14.096 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:14.097 INFO:teuthology.orchestra.run.smithi055.stdout:Total 220 kB/s | 131 kB 00:00 2023-12-06T03:19:14.097 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:14.104 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:14.104 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:14.166 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:14.169 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:14.561 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:15.304 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:15.532 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:15.533 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:15.533 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:15.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-cephfs 2023-12-06T03:19:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:15.857 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:15.857 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:16.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install libcephfs2 2023-12-06T03:19:16.510 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:28 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:17.074 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:26 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:17.965 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:18.191 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:18.192 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:18.192 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:18.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rbd 2023-12-06T03:19:18.697 INFO:teuthology.orchestra.run.smithi055.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:18.946 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:18.948 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:18.948 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:19.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install libcephfs-devel 2023-12-06T03:19:19.177 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:31 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:20.073 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:29 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:20.622 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:20.849 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:20.851 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2023-12-06T03:19:20.851 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:20.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-fuse 2023-12-06T03:19:21.826 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:33 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:21.950 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:21.951 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 24 k 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 94 k 2023-12-06T03:19:21.952 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:21.997 INFO:teuthology.orchestra.run.smithi055.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 541 kB/s | 24 kB 00:00 2023-12-06T03:19:21.998 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:21.998 INFO:teuthology.orchestra.run.smithi055.stdout:Total 525 kB/s | 24 kB 00:00 2023-12-06T03:19:21.999 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:22.002 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:22.002 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:22.005 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:22.009 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:22.064 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:22.604 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:23.349 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:23.349 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:23.349 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:23.349 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:23.349 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:23.349 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:23.494 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:23.494 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:19:23.495 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:23.496 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 89 k 2023-12-06T03:19:23.496 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 298 k 2023-12-06T03:19:23.496 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:19:23.532 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install librados2 2023-12-06T03:19:23.541 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 1.9 MB/s | 89 kB 00:00 2023-12-06T03:19:23.541 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:23.542 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.9 MB/s | 89 kB 00:00 2023-12-06T03:19:23.542 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:19:23.551 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:19:23.551 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:19:23.601 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:19:23.604 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:19:23.720 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:19:23.937 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:24.446 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:24.534 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:33 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:25.128 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:25.128 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:19:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:25.129 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:25.129 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:25.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-mirror 2023-12-06T03:19:26.179 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:26.214 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:38 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:26.425 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:26.426 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:26.426 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:26.543 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install librbd1 2023-12-06T03:19:27.539 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:36 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:27.883 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 3.7 M 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 18 M 2023-12-06T03:19:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:19:27.970 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 45 MB/s | 3.7 MB 00:00 2023-12-06T03:19:27.971 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:27.971 INFO:teuthology.orchestra.run.smithi175.stdout:Total 44 MB/s | 3.7 MB 00:00 2023-12-06T03:19:27.971 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:19:27.981 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:19:27.981 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:19:28.045 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:19:28.048 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:19:28.512 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:19:28.712 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:29.117 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:29.159 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:29.413 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:29.414 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:29.415 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:29.534 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install python3-rados 2023-12-06T03:19:29.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:29.822 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:29.822 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:19:29.822 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:29.822 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:29.822 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:30.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-nbd 2023-12-06T03:19:30.528 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:39 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:30.917 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:02:42 ago on Wed 06 Dec 2023 03:16:48 AM UTC. 2023-12-06T03:19:32.197 INFO:teuthology.orchestra.run.smithi055.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:32.447 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:32.449 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:32.449 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:32.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install python3-rgw 2023-12-06T03:19:32.637 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2023-12-06T03:19:32.638 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 173 k 2023-12-06T03:19:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 649 k 2023-12-06T03:19:32.639 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2023-12-06T03:19:32.695 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 3.0 MB/s | 173 kB 00:00 2023-12-06T03:19:32.695 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:32.695 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.0 MB/s | 173 kB 00:00 2023-12-06T03:19:32.696 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2023-12-06T03:19:32.707 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2023-12-06T03:19:32.707 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2023-12-06T03:19:32.758 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2023-12-06T03:19:32.761 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2023-12-06T03:19:32.879 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2023-12-06T03:19:33.097 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:33.391 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:33.566 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:42 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:34.061 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2023-12-06T03:19:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:34.061 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:19:34.061 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2023-12-06T03:19:34.277 DEBUG:teuthology.parallel:result is None 2023-12-06T03:19:35.186 INFO:teuthology.orchestra.run.smithi055.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:35.435 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:35.437 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:35.437 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:35.552 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install python3-cephfs 2023-12-06T03:19:36.549 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:45 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:38.191 INFO:teuthology.orchestra.run.smithi055.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:38.446 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:38.447 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:38.447 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:38.566 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install python3-rbd 2023-12-06T03:19:39.562 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:48 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:41.175 INFO:teuthology.orchestra.run.smithi055.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2023-12-06T03:19:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:41.428 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-12-06T03:19:41.428 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:41.546 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install rbd-fuse 2023-12-06T03:19:42.547 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:51 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:44.432 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:19:44.433 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:44.434 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:19:44.434 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:44.434 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 89 k 2023-12-06T03:19:44.434 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 298 k 2023-12-06T03:19:44.434 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:44.504 INFO:teuthology.orchestra.run.smithi055.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 1.2 MB/s | 89 kB 00:00 2023-12-06T03:19:44.505 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:44.505 INFO:teuthology.orchestra.run.smithi055.stdout:Total 1.2 MB/s | 89 kB 00:00 2023-12-06T03:19:44.505 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:44.517 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:44.517 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:44.574 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:44.579 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:44.713 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:44.879 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:45.203 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:45.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:45.947 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:45.947 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:45.947 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:45.948 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:45.948 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:46.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install rbd-mirror 2023-12-06T03:19:47.185 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:56 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2023-12-06T03:19:49.043 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 3.7 M 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 18 M 2023-12-06T03:19:49.044 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:49.140 INFO:teuthology.orchestra.run.smithi055.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 39 MB/s | 3.7 MB 00:00 2023-12-06T03:19:49.141 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:49.141 INFO:teuthology.orchestra.run.smithi055.stdout:Total 38 MB/s | 3.7 MB 00:00 2023-12-06T03:19:49.142 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:49.156 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:49.156 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:49.227 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:49.230 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:49.742 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:49.920 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:50.439 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:51.178 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:51.178 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:51.178 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:51.179 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:51.179 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:51.179 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:51.424 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install rbd-nbd 2023-12-06T03:19:52.417 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:03:01 ago on Wed 06 Dec 2023 03:16:51 AM UTC. 2023-12-06T03:19:54.278 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-12-06T03:19:54.278 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2023-12-06T03:19:54.279 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:54.280 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 173 k 2023-12-06T03:19:54.280 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 649 k 2023-12-06T03:19:54.280 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-12-06T03:19:54.326 INFO:teuthology.orchestra.run.smithi055.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 3.7 MB/s | 173 kB 00:00 2023-12-06T03:19:54.327 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-12-06T03:19:54.327 INFO:teuthology.orchestra.run.smithi055.stdout:Total 3.6 MB/s | 173 kB 00:00 2023-12-06T03:19:54.327 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-12-06T03:19:54.340 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-12-06T03:19:54.340 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-12-06T03:19:54.397 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-12-06T03:19:54.400 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-12-06T03:19:54.522 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-12-06T03:19:54.878 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:55.195 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:55.996 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2023-12-06T03:19:55.996 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:55.996 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-12-06T03:19:55.996 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2023-12-06T03:19:55.997 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-12-06T03:19:55.997 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-12-06T03:19:56.208 DEBUG:teuthology.parallel:result is None 2023-12-06T03:19:56.209 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2023-12-06T03:19:56.209 INFO:teuthology.packaging:ref: None 2023-12-06T03:19:56.209 INFO:teuthology.packaging:tag: v16.2.4 2023-12-06T03:19:56.209 INFO:teuthology.packaging:branch: None 2023-12-06T03:19:56.209 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:19:56.400 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:19:56.400 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:19:56.569 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-06T03:19:56.599 INFO:teuthology.orchestra.run.smithi055.stdout:16.2.4-0.el8 2023-12-06T03:19:56.600 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2023-12-06T03:19:56.600 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2023-12-06T03:19:56.601 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2023-12-06T03:19:56.601 INFO:teuthology.packaging:ref: None 2023-12-06T03:19:56.601 INFO:teuthology.packaging:tag: v16.2.4 2023-12-06T03:19:56.602 INFO:teuthology.packaging:branch: None 2023-12-06T03:19:56.602 INFO:teuthology.packaging:sha1: d03fca4f35d43f61b4679936aef7bd2f01cced0b 2023-12-06T03:19:56.804 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:19:56.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2023-12-06T03:19:57.027 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-06T03:19:57.054 INFO:teuthology.orchestra.run.smithi175.stdout:16.2.4-0.el8 2023-12-06T03:19:57.054 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2023-12-06T03:19:57.055 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2023-12-06T03:19:57.055 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-06T03:19:57.055 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:19:57.055 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-06T03:19:57.092 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:19:57.092 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-06T03:19:57.126 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-06T03:19:57.127 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:19:57.127 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2023-12-06T03:19:57.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-06T03:19:57.241 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:19:57.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2023-12-06T03:19:57.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-06T03:19:57.342 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-06T03:19:57.343 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:19:57.343 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-06T03:19:57.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-06T03:19:57.454 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:19:57.454 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-06T03:19:57.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-06T03:19:57.553 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-06T03:19:57.553 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:19:57.553 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2023-12-06T03:19:57.588 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-06T03:19:57.663 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:19:57.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2023-12-06T03:19:57.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-06T03:19:57.761 INFO:teuthology.run_tasks:Running task print... 2023-12-06T03:19:57.772 INFO:teuthology.task.print:**** done install task... 2023-12-06T03:19:57.773 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-06T03:19:57.846 INFO:tasks.cephadm:Config: {'avoid_pacific_features': True, 'cephadm_branch': 'v16.2.4', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'docker.io/ceph/ceph:v16.2.4', 'roleless': True, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd03fca4f35d43f61b4679936aef7bd2f01cced0b'} 2023-12-06T03:19:57.846 INFO:tasks.cephadm:Cluster image is docker.io/ceph/ceph:v16.2.4 2023-12-06T03:19:57.846 INFO:tasks.cephadm:Cluster fsid is 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:19:57.847 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-06T03:19:57.847 INFO:tasks.cephadm:No mon roles; fabricating mons 2023-12-06T03:19:57.847 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi055': '172.21.15.55', 'mon.smithi175': '172.21.15.175'} 2023-12-06T03:19:57.847 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-06T03:19:57.847 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname $(hostname -s) 2023-12-06T03:19:57.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2023-12-06T03:19:57.912 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v16.2.4)... 2023-12-06T03:19:57.912 DEBUG:teuthology.orchestra.run.smithi055:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-06T03:19:58.152 INFO:teuthology.orchestra.run.smithi055.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Dec 6 03:19 /home/ubuntu/cephtest/cephadm 2023-12-06T03:19:58.152 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-06T03:19:58.294 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Dec 6 03:19 /home/ubuntu/cephtest/cephadm 2023-12-06T03:19:58.294 DEBUG:teuthology.orchestra.run.smithi055:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-06T03:19:58.321 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-06T03:19:58.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph 2023-12-06T03:19:58.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2023-12-06T03:19:58.424 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /etc/ceph 2023-12-06T03:19:58.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2023-12-06T03:19:58.500 INFO:tasks.cephadm:Writing seed config... 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-06T03:19:58.501 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-12-06T03:19:58.502 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-06T03:19:58.503 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-06T03:19:58.504 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:19:58.504 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-06T03:19:58.535 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5555246c-93e6-11ee-95a2-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-06T03:19:58.535 DEBUG:teuthology.orchestra.run.smithi055:mon.smithi055> sudo journalctl -f -n 0 -u ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055.service 2023-12-06T03:19:58.577 INFO:tasks.cephadm:Bootstrapping... 2023-12-06T03:19:58.578 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 -v bootstrap --fsid 5555246c-93e6-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.55 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-06T03:19:58.611 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:-- Logs begin at Wed 2023-12-06 03:07:20 UTC. -- 2023-12-06T03:19:58.888 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2023-12-06T03:19:59.721 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: 4.6.1 2023-12-06T03:19:59.731 INFO:teuthology.orchestra.run.smithi055.stderr:Verifying podman|docker is present... 2023-12-06T03:19:59.731 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2023-12-06T03:19:59.756 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: 4.6.1 2023-12-06T03:19:59.769 INFO:teuthology.orchestra.run.smithi055.stderr:Verifying lvm2 is present... 2023-12-06T03:19:59.769 INFO:teuthology.orchestra.run.smithi055.stderr:Verifying time synchronization is in place... 2023-12-06T03:19:59.769 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled chrony.service 2023-12-06T03:19:59.777 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2023-12-06T03:19:59.778 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active chrony.service 2023-12-06T03:19:59.785 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: inactive 2023-12-06T03:19:59.785 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled chronyd.service 2023-12-06T03:19:59.792 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: enabled 2023-12-06T03:19:59.793 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active chronyd.service 2023-12-06T03:19:59.799 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: active 2023-12-06T03:19:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:Unit chronyd.service is enabled and running 2023-12-06T03:19:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:Repeating the final host check... 2023-12-06T03:19:59.800 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2023-12-06T03:19:59.823 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: 4.6.1 2023-12-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi055.stderr:podman|docker (/bin/podman) is present 2023-12-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl is present 2023-12-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi055.stderr:lvcreate is present 2023-12-06T03:19:59.830 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled chrony.service 2023-12-06T03:19:59.837 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2023-12-06T03:19:59.837 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active chrony.service 2023-12-06T03:19:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: inactive 2023-12-06T03:19:59.844 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled chronyd.service 2023-12-06T03:19:59.850 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: enabled 2023-12-06T03:19:59.850 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active chronyd.service 2023-12-06T03:19:59.856 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: active 2023-12-06T03:19:59.857 INFO:teuthology.orchestra.run.smithi055.stderr:Unit chronyd.service is enabled and running 2023-12-06T03:19:59.857 INFO:teuthology.orchestra.run.smithi055.stderr:Host looks OK 2023-12-06T03:19:59.857 INFO:teuthology.orchestra.run.smithi055.stderr:Cluster fsid: 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:19:59.858 INFO:teuthology.orchestra.run.smithi055.stderr:Acquiring lock 140226314443240 on /run/cephadm/5555246c-93e6-11ee-95a2-87774f69a715.lock 2023-12-06T03:19:59.858 INFO:teuthology.orchestra.run.smithi055.stderr:Lock 140226314443240 acquired on /run/cephadm/5555246c-93e6-11ee-95a2-87774f69a715.lock 2023-12-06T03:19:59.858 INFO:teuthology.orchestra.run.smithi055.stderr:Verifying IP 172.21.15.55 port 3300 ... 2023-12-06T03:19:59.859 INFO:teuthology.orchestra.run.smithi055.stderr:Verifying IP 172.21.15.55 port 6789 ... 2023-12-06T03:19:59.859 INFO:teuthology.orchestra.run.smithi055.stderr:Base mon IP is 172.21.15.55, final addrv is [v2:172.21.15.55:3300,v1:172.21.15.55:6789] 2023-12-06T03:19:59.859 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /sbin/ip route ls 2023-12-06T03:19:59.861 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.55 metric 100 2023-12-06T03:19:59.861 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.55 metric 100 2023-12-06T03:19:59.862 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /sbin/ip -6 route ls 2023-12-06T03:19:59.864 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2023-12-06T03:19:59.865 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-12-06T03:19:59.865 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2023-12-06T03:19:59.865 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /sbin/ip -6 addr ls 2023-12-06T03:19:59.868 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-06T03:19:59.868 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: inet6 ::1/128 scope host 2023-12-06T03:19:59.868 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2023-12-06T03:19:59.868 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: 3: ens1f0: mtu 1500 state UP qlen 1000 2023-12-06T03:19:59.868 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-12-06T03:19:59.868 INFO:teuthology.orchestra.run.smithi055.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2023-12-06T03:19:59.869 INFO:teuthology.orchestra.run.smithi055.stderr:Mon IP 172.21.15.55 is in CIDR network 172.21.0.0/20 2023-12-06T03:19:59.869 INFO:teuthology.orchestra.run.smithi055.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-06T03:19:59.871 INFO:teuthology.orchestra.run.smithi055.stderr:Pulling container image docker.io/ceph/ceph:v16.2.4... 2023-12-06T03:19:59.871 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman pull docker.io/ceph/ceph:v16.2.4 2023-12-06T03:19:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: Trying to pull docker.io/ceph/ceph:v16.2.4... 2023-12-06T03:20:06.499 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: Getting image source signatures 2023-12-06T03:20:06.513 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: Copying blob sha256:551101690791055467b1840be133b2e6b029d263771bc6c46a96bb7acdacac39 2023-12-06T03:20:06.513 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: Copying blob sha256:7a0437f04f83f084b7ed68ad9c4a4947e12fc4e1b006b38129bac89114ec3621 2023-12-06T03:20:19.488 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: Copying config sha256:8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2023-12-06T03:20:19.538 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: Writing manifest to image destination 2023-12-06T03:20:19.819 INFO:teuthology.orchestra.run.smithi055.stderr:/bin/podman: 8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2023-12-06T03:20:19.835 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --version 2023-12-06T03:20:20.416 INFO:teuthology.orchestra.run.smithi055.stderr:ceph: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2023-12-06T03:20:20.770 INFO:teuthology.orchestra.run.smithi055.stderr:Ceph version: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2023-12-06T03:20:20.770 INFO:teuthology.orchestra.run.smithi055.stderr:Extracting ceph user uid/gid from container image... 2023-12-06T03:20:20.770 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 -c %u %g /var/lib/ceph 2023-12-06T03:20:21.042 INFO:teuthology.orchestra.run.smithi055.stderr:stat: 167 167 2023-12-06T03:20:21.361 INFO:teuthology.orchestra.run.smithi055.stderr:Creating initial keys... 2023-12-06T03:20:21.361 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2023-12-06T03:20:21.634 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-authtool: AQB16G9lDqf8JBAAFUyHgYcAtJrUfFqVZ2b3oQ== 2023-12-06T03:20:21.958 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2023-12-06T03:20:22.250 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-authtool: AQB26G9lBw23DRAA5/ud8Sn1vvlXNKj0Bvsl/g== 2023-12-06T03:20:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2023-12-06T03:20:22.859 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-authtool: AQB26G9l0VyiMhAAvtp15C6n3d8+U9nR44SvYw== 2023-12-06T03:20:23.243 INFO:teuthology.orchestra.run.smithi055.stderr:Creating initial monmap... 2023-12-06T03:20:23.243 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/monmaptool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /tmp/ceph-tmpagt6yu4v:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --create --clobber --fsid 5555246c-93e6-11ee-95a2-87774f69a715 --addv smithi055 [v2:172.21.15.55:3300,v1:172.21.15.55:6789] /tmp/monmap 2023-12-06T03:20:23.509 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-06T03:20:23.509 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:23.509 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-06T03:20:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:monmaptool for smithi055 [v2:172.21.15.55:3300,v1:172.21.15.55:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-06T03:20:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/monmaptool: set fsid to 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-06T03:20:23.845 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:20:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:Creating mon... 2023-12-06T03:20:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-mon --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/mon.smithi055:/var/lib/ceph/mon/ceph-smithi055:z -v /tmp/ceph-tmp41ikgqrj:/tmp/keyring:z -v /tmp/ceph-tmpagt6yu4v:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --mkfs -i smithi055 --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.125+0000 7f0ee14df700 0 set uid:gid to 167:167 (ceph:ceph) 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.126+0000 7f0ee14df700 1 imported monmap: 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: epoch 0 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: fsid 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: last_changed 2023-12-06T03:20:23.500496+0000 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: created 2023-12-06T03:20:23.500496+0000 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2023-12-06T03:20:24.128 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: election_strategy: 1 2023-12-06T03:20:24.129 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.smithi055 2023-12-06T03:20:24.129 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.129 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.126+0000 7f0ee14df700 0 /usr/bin/ceph-mon: set fsid to 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:24.133 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: RocksDB version: 6.8.1 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Compile date May 13 2021 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: DB SUMMARY 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi055/store.db dir, Total Num: 0, files: 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi055/store.db: 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.error_if_exists: 0 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.create_if_missing: 1 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.paranoid_checks: 1 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.env: 0x556ebfbe61c0 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.fs: Posix File System 2023-12-06T03:20:24.134 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.info_log: 0x556ec12f8ec0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_file_opening_threads: 16 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.statistics: (nil) 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.use_fsync: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_log_file_size: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.log_file_time_to_roll: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.keep_log_file_num: 1000 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.recycle_log_file_num: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.allow_fallocate: 1 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.allow_mmap_reads: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.allow_mmap_writes: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.use_direct_reads: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.create_missing_column_families: 0 2023-12-06T03:20:24.135 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.db_log_dir: 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi055/store.db 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.table_cache_numshardbits: 6 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_subcompactions: 1 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_background_flushes: -1 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.WAL_ttl_seconds: 0 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.WAL_size_limit_MB: 0 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.is_fd_close_on_exec: 1 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.advise_random_on_open: 1 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.db_write_buffer_size: 0 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.write_buffer_manager: 0x556ec12ff530 2023-12-06T03:20:24.136 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.use_adaptive_mutex: 0 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.rate_limiter: (nil) 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.wal_recovery_mode: 2 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.enable_thread_tracking: 0 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.enable_pipelined_write: 0 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.unordered_write: 0 2023-12-06T03:20:24.137 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-06T03:20:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-06T03:20:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-06T03:20:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-06T03:20:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.row_cache: None 2023-12-06T03:20:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.wal_filter: None 2023-12-06T03:20:24.156 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.allow_ingest_behind: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.preserve_deletes: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.two_write_queues: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.manual_wal_flush: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.atomic_flush: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.persist_stats_to_disk: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.write_dbid_to_manifest: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.log_readahead_size: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_background_jobs: 2 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_background_compactions: -1 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.delayed_write_rate : 16777216 2023-12-06T03:20:24.157 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_total_wal_size: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.stats_dump_period_sec: 600 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.stats_persist_period_sec: 600 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.max_open_files: -1 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.bytes_per_sync: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.wal_bytes_per_sync: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.strict_bytes_per_sync: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Options.compaction_readahead_size: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Compression algorithms supported: 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kZSTD supported: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kXpressCompression supported: 0 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kLZ4HCCompression supported: 1 2023-12-06T03:20:24.158 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kLZ4Compression supported: 1 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kBZip2Compression supported: 0 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kZlibCompression supported: 1 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: kSnappyCompression supported: 1 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.127+0000 7f0ee14df700 4 rocksdb: Fast CRC32 supported: Supported on x86 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.128+0000 7f0ee14df700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi055/store.db/MANIFEST-000001 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.merge_operator: 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_filter: None 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_filter_factory: None 2023-12-06T03:20:24.159 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.memtable_factory: SkipListFactory 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.table_factory: BlockBasedTable 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556ec1210120) 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: index_type: 0 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: index_shortening: 1 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: checksum: 1 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_cache: 0x556ec1246d10 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2023-12-06T03:20:24.160 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_cache_options: 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: capacity : 536870912 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_size: 4096 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: partition_filters: 0 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2023-12-06T03:20:24.161 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: verify_compression: 0 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: format_version: 2 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: block_align: 0 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.write_buffer_size: 33554432 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_write_buffer_number: 2 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression: NoCompression 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression: Disabled 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.prefix_extractor: nullptr 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.num_levels: 7 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-06T03:20:24.162 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression_opts.window_bits: -14 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression_opts.level: 32767 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression_opts.strategy: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compression_opts.enabled: false 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-06T03:20:24.163 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.target_file_size_base: 67108864 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.target_file_size_multiplier: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.arena_block_size: 4194304 2023-12-06T03:20:24.164 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.disable_auto_compactions: 0 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.table_properties_collectors: 2023-12-06T03:20:24.165 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.inplace_update_support: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.inplace_update_num_locks: 10000 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.memtable_huge_page_size: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.bloom_locality: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.max_successive_merges: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.optimize_filters_for_hits: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.paranoid_file_checks: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.force_consistency_checks: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.report_bg_io_stats: 0 2023-12-06T03:20:24.166 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.ttl: 2592000 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.130+0000 7f0ee14df700 4 rocksdb: Options.periodic_compaction_seconds: 0 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.131+0000 7f0ee14df700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi055/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.131+0000 7f0ee14df700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.132+0000 7f0ee14df700 4 rocksdb: DB pointer 0x556ec130b800 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.132+0000 7f0eca378700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.132+0000 7f0eca378700 4 rocksdb: [db_impl/db_impl.cc:851] 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-06T03:20:24.167 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:24.168 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.169 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: 2023-12-06T03:20:24.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.134+0000 7f0ee14df700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2023-12-06T03:20:24.171 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.134+0000 7f0ee14df700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2023-12-06T03:20:24.171 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph-mon: debug 2023-12-06T03:20:24.134+0000 7f0ee14df700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-smithi055 for mon.smithi055 2023-12-06T03:20:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:create mon.smithi055 on 2023-12-06T03:20:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:24.514 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl disable ceph.target 2023-12-06T03:20:24.522 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-06T03:20:24.621 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl enable ceph.target 2023-12-06T03:20:24.628 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-06T03:20:24.719 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl start ceph.target 2023-12-06T03:20:24.727 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl enable ceph-5555246c-93e6-11ee-95a2-87774f69a715.target 2023-12-06T03:20:24.733 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5555246c-93e6-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-5555246c-93e6-11ee-95a2-87774f69a715.target. 2023-12-06T03:20:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-5555246c-93e6-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-5555246c-93e6-11ee-95a2-87774f69a715.target. 2023-12-06T03:20:24.824 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715.target 2023-12-06T03:20:24.831 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl daemon-reload 2023-12-06T03:20:24.926 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055 2023-12-06T03:20:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl reset-failed ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055 2023-12-06T03:20:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Failed to reset failed state of unit ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055.service: Unit ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055.service not loaded. 2023-12-06T03:20:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl enable ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055 2023-12-06T03:20:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Created symlink /etc/systemd/system/ceph-5555246c-93e6-11ee-95a2-87774f69a715.target.wants/ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055.service → /etc/systemd/system/ceph-5555246c-93e6-11ee-95a2-87774f69a715@.service. 2023-12-06T03:20:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055 2023-12-06T03:20:25.074 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:25 smithi055 systemd[1]: Starting Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715... 2023-12-06T03:20:25.422 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled firewalld.service 2023-12-06T03:20:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: disabled 2023-12-06T03:20:25.433 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active firewalld.service 2023-12-06T03:20:25.442 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: inactive 2023-12-06T03:20:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:firewalld.service is not enabled 2023-12-06T03:20:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:Not possible to enable service . firewalld.service is not available 2023-12-06T03:20:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:Not possible to open ports <[]>. firewalld.service is not available 2023-12-06T03:20:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for mon to start... 2023-12-06T03:20:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for mon... 2023-12-06T03:20:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/mon.smithi055:/var/lib/ceph/mon/ceph-smithi055:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status 2023-12-06T03:20:25.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:25 smithi055 bash[111335]: 7f509929bf4e9ec9614ff14748a863f6e02fe9da2fe2d67a98ef18aa0e906248 2023-12-06T03:20:25.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:25 smithi055 systemd[1]: Started Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715. 2023-12-06T03:20:25.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.463471+0000 mon.smithi055 (mon.0) 0 : [INF] mkfs 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:25.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.444012+0000 mon.smithi055 (mon.0) 1 : cluster [INF] mon.smithi055 is new leader, mons smithi055 in quorum (ranks 0) 2023-12-06T03:20:25.866 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.864+0000 7fe84f291700 1 Processor -- start 2023-12-06T03:20:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.864+0000 7fe84f291700 1 -- start start 2023-12-06T03:20:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.864+0000 7fe84f291700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe8480fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.864+0000 7fe84f291700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8480fb490 con 0x7fe8480fab80 2023-12-06T03:20:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe84d02d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe8480fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:25.868 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe84d02d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe8480fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58382/0 (socket says 172.21.15.55:58382) 2023-12-06T03:20:25.868 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe84d02d700 1 -- 172.21.15.55:0/3690902051 learned_addr learned my addr 172.21.15.55:0/3690902051 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:25.868 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe84d02d700 1 -- 172.21.15.55:0/3690902051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480fb5d0 con 0x7fe8480fab80 2023-12-06T03:20:25.868 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe84d02d700 1 --2- 172.21.15.55:0/3690902051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe8480fafa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fe84400b5f0 tx=0x7fe844006ee0).ready entity=mon.0 client_cookie=4e211abc864186ba server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:25.868 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe837fff700 1 -- 172.21.15.55:0/3690902051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe84400d600 con 0x7fe8480fab80 2023-12-06T03:20:25.868 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe837fff700 1 -- 172.21.15.55:0/3690902051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400d760 con 0x7fe8480fab80 2023-12-06T03:20:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.865+0000 7fe837fff700 1 -- 172.21.15.55:0/3690902051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe84400da30 con 0x7fe8480fab80 2023-12-06T03:20:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 -- 172.21.15.55:0/3690902051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 msgr2=0x7fe8480fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 --2- 172.21.15.55:0/3690902051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe8480fafa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fe84400b5f0 tx=0x7fe844006ee0).stop 2023-12-06T03:20:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 -- 172.21.15.55:0/3690902051 shutdown_connections 2023-12-06T03:20:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 --2- 172.21.15.55:0/3690902051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe8480fafa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 -- 172.21.15.55:0/3690902051 >> 172.21.15.55:0/3690902051 conn(0x7fe8480f63c0 msgr2=0x7fe8480f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 -- 172.21.15.55:0/3690902051 shutdown_connections 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.866+0000 7fe84f291700 1 -- 172.21.15.55:0/3690902051 wait complete. 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84f291700 1 Processor -- start 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84f291700 1 -- start start 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84f291700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe84810f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84f291700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84810ad60 con 0x7fe8480fab80 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84d02d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe84810f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84d02d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe84810f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58388/0 (socket says 172.21.15.55:58388) 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.867+0000 7fe84d02d700 1 -- 172.21.15.55:0/2366707643 learned_addr learned my addr 172.21.15.55:0/2366707643 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe84d02d700 1 -- 172.21.15.55:0/2366707643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84400b200 con 0x7fe8480fab80 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe84d02d700 1 --2- 172.21.15.55:0/2366707643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe84810f370 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe844006c70 tx=0x7fe844016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe844011b40 con 0x7fe8480fab80 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe844011ca0 con 0x7fe8480fab80 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe84401d430 con 0x7fe8480fab80 2023-12-06T03:20:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe848070450 con 0x7fe8480fab80 2023-12-06T03:20:25.872 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84810f860 con 0x7fe8480fab80 2023-12-06T03:20:25.872 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.868+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fe84400f070 con 0x7fe8480fab80 2023-12-06T03:20:25.872 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.869+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe844027460 con 0x7fe8480fab80 2023-12-06T03:20:25.872 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.870+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe848045ad0 con 0x7fe8480fab80 2023-12-06T03:20:25.874 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.872+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fe844016070 con 0x7fe8480fab80 2023-12-06T03:20:25.948 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.945+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe8480fbec0 con 0x7fe8480fab80 2023-12-06T03:20:25.948 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.945+0000 7fe835ffb700 1 -- 172.21.15.55:0/2366707643 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+324 (secure 0 0 0) 0x7fe844035160 con 0x7fe8480fab80 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: cluster: 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: id: 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: health: HEALTH_OK 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: services: 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon: 1 daemons, quorum smithi055 (age 0.48383s) 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mgr: no daemons active 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: data: 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: pgs: 2023-12-06T03:20:25.949 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 msgr2=0x7fe84810f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 --2- 172.21.15.55:0/2366707643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe84810f370 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe844006c70 tx=0x7fe844016000).stop 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 shutdown_connections 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 --2- 172.21.15.55:0/2366707643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8480fab80 0x7fe84810f370 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 >> 172.21.15.55:0/2366707643 conn(0x7fe8480f63c0 msgr2=0x7fe8480fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 shutdown_connections 2023-12-06T03:20:25.950 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:25.946+0000 7fe84f291700 1 -- 172.21.15.55:0/2366707643 wait complete. 2023-12-06T03:20:26.328 INFO:teuthology.orchestra.run.smithi055.stderr:mon is available 2023-12-06T03:20:26.328 INFO:teuthology.orchestra.run.smithi055.stderr:Assimilating anything we can from ceph.conf... 2023-12-06T03:20:26.328 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z -v /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/mon.smithi055:/var/lib/ceph/mon/ceph-smithi055:z docker.io/ceph/ceph:v16.2.4 config assimilate-conf -i /var/lib/ceph/mon/ceph-smithi055/config 2023-12-06T03:20:26.545 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.462673+0000 mon.smithi055 (mon.0) 2 : cluster [INF] mon.smithi055 is new leader, mons smithi055 in quorum (ranks 0) 2023-12-06T03:20:26.545 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.463013+0000 mon.smithi055 (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {smithi055=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0]} 2023-12-06T03:20:26.545 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.467334+0000 mon.smithi055 (mon.0) 4 : cluster [DBG] fsmap 2023-12-06T03:20:26.545 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.472503+0000 mon.smithi055 (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2023-12-06T03:20:26.545 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: cluster 2023-12-06T03:20:25.473972+0000 mon.smithi055 (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2023-12-06T03:20:26.545 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: audit 2023-12-06T03:20:25.946465+0000 mon.smithi055 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.55:0/2366707643' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-06T03:20:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.791+0000 7fdbf5820700 1 Processor -- start 2023-12-06T03:20:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.791+0000 7fdbf5820700 1 -- start start 2023-12-06T03:20:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbf5820700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf00fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbf5820700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00fc5c0 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf00fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf00fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58390/0 (socket says 172.21.15.55:58390) 2023-12-06T03:20:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbeeffd700 1 -- 172.21.15.55:0/3181715211 learned_addr learned my addr 172.21.15.55:0/3181715211 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbeeffd700 1 -- 172.21.15.55:0/3181715211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00fc700 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.796 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.792+0000 7fdbeeffd700 1 --2- 172.21.15.55:0/3181715211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf00fc0d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fdbe000b5f0 tx=0x7fdbe0006ee0).ready entity=mon.0 client_cookie=8e5f49a26a3bce0e server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:26.796 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbee7fc700 1 -- 172.21.15.55:0/3181715211 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbe000d600 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.796 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbee7fc700 1 -- 172.21.15.55:0/3181715211 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000d760 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.796 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbee7fc700 1 -- 172.21.15.55:0/3181715211 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbe000da30 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.796 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbf5820700 1 -- 172.21.15.55:0/3181715211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 msgr2=0x7fdbf00fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:26.796 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbf5820700 1 --2- 172.21.15.55:0/3181715211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf00fc0d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fdbe000b5f0 tx=0x7fdbe0006ee0).stop 2023-12-06T03:20:26.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbf5820700 1 -- 172.21.15.55:0/3181715211 shutdown_connections 2023-12-06T03:20:26.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbf5820700 1 --2- 172.21.15.55:0/3181715211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf00fc0d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:26.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.793+0000 7fdbf5820700 1 -- 172.21.15.55:0/3181715211 >> 172.21.15.55:0/3181715211 conn(0x7fdbf00f74b0 msgr2=0x7fdbf00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:26.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.794+0000 7fdbf5820700 1 -- 172.21.15.55:0/3181715211 shutdown_connections 2023-12-06T03:20:26.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.794+0000 7fdbf5820700 1 -- 172.21.15.55:0/3181715211 wait complete. 2023-12-06T03:20:26.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.794+0000 7fdbf5820700 1 Processor -- start 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbf5820700 1 -- start start 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbf5820700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf010a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbf5820700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf010a770 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf010a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbeeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf010a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58406/0 (socket says 172.21.15.55:58406) 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbeeffd700 1 -- 172.21.15.55:0/3402558790 learned_addr learned my addr 172.21.15.55:0/3402558790 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbeeffd700 1 -- 172.21.15.55:0/3402558790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe000b200 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.795+0000 7fdbeeffd700 1 --2- 172.21.15.55:0/3402558790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf010a280 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7fdbe0006c70 tx=0x7fdbe0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbe0011b40 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0011ca0 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf010a970 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdbe001d430 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf010ad80 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fdbe000f070 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.796+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fdbe0027460 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.800 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.797+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf0045ad0 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.801 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.799+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fdbe0016070 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.869+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdbf00fcff0 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.875+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 7 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fdbe001d590 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.875+0000 7fdbe7fff700 1 -- 172.21.15.55:0/3402558790 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+464 (secure 0 0 0) 0x7fdbe00276f0 con 0x7fdbf00fbcb0 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: [global] 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: fsid = 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.55:3300,v1:172.21.15.55:6789] 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: [mgr] 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: [osd] 2023-12-06T03:20:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_osd_backfillfull_ratio = 0.85 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_osd_full_ratio = 0.9 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: mon_osd_nearfull_ratio = 0.8 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: osd_memory_target_autotune = true 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 msgr2=0x7fdbf010a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 --2- 172.21.15.55:0/3402558790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf010a280 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7fdbe0006c70 tx=0x7fdbe0016000).stop 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 shutdown_connections 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 --2- 172.21.15.55:0/3402558790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbf00fbcb0 0x7fdbf010a280 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 >> 172.21.15.55:0/3402558790 conn(0x7fdbf00f74b0 msgr2=0x7fdbf00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 shutdown_connections 2023-12-06T03:20:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:26.876+0000 7fdbf5820700 1 -- 172.21.15.55:0/3402558790 wait complete. 2023-12-06T03:20:27.228 INFO:teuthology.orchestra.run.smithi055.stderr:Generating new minimal ceph.conf... 2023-12-06T03:20:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z -v /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/mon.smithi055:/var/lib/ceph/mon/ceph-smithi055:z docker.io/ceph/ceph:v16.2.4 config generate-minimal-conf -o /var/lib/ceph/mon/ceph-smithi055/config 2023-12-06T03:20:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.610+0000 7f0950cb8700 1 Processor -- start 2023-12-06T03:20:27.613 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.610+0000 7f0950cb8700 1 -- start start 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.611+0000 7f0950cb8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c0fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.611+0000 7f0950cb8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0fee20 con 0x7f094c070460 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.611+0000 7f094a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c0fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.611+0000 7f094a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c0fe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58418/0 (socket says 172.21.15.55:58418) 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.611+0000 7f094a59c700 1 -- 172.21.15.55:0/581088214 learned_addr learned my addr 172.21.15.55:0/581088214 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.611+0000 7f094a59c700 1 -- 172.21.15.55:0/581088214 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0fef60 con 0x7f094c070460 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f094a59c700 1 --2- 172.21.15.55:0/581088214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c0fe930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f093c00b5f0 tx=0x7f093c006ee0).ready entity=mon.0 client_cookie=920e8af7a955c3f server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f094959a700 1 -- 172.21.15.55:0/581088214 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f093c00d600 con 0x7f094c070460 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f094959a700 1 -- 172.21.15.55:0/581088214 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f093c00dbd0 con 0x7f094c070460 2023-12-06T03:20:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f094959a700 1 -- 172.21.15.55:0/581088214 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f093c011c00 con 0x7f094c070460 2023-12-06T03:20:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f0950cb8700 1 -- 172.21.15.55:0/581088214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 msgr2=0x7f094c0fe930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f0950cb8700 1 --2- 172.21.15.55:0/581088214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c0fe930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f093c00b5f0 tx=0x7f093c006ee0).stop 2023-12-06T03:20:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f0950cb8700 1 -- 172.21.15.55:0/581088214 shutdown_connections 2023-12-06T03:20:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f0950cb8700 1 --2- 172.21.15.55:0/581088214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c0fe930 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.612+0000 7f0950cb8700 1 -- 172.21.15.55:0/581088214 >> 172.21.15.55:0/581088214 conn(0x7f094c0f63b0 msgr2=0x7f094c0f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.613+0000 7f0950cb8700 1 -- 172.21.15.55:0/581088214 shutdown_connections 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.613+0000 7f0950cb8700 1 -- 172.21.15.55:0/581088214 wait complete. 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.613+0000 7f0950cb8700 1 Processor -- start 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.613+0000 7f0950cb8700 1 -- start start 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f0950cb8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c10aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f0950cb8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c10af50 con 0x7f094c070460 2023-12-06T03:20:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f094a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c10aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f094a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c10aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58426/0 (socket says 172.21.15.55:58426) 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f094a59c700 1 -- 172.21.15.55:0/1440201877 learned_addr learned my addr 172.21.15.55:0/1440201877 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f094a59c700 1 -- 172.21.15.55:0/1440201877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c00b200 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f094a59c700 1 --2- 172.21.15.55:0/1440201877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c10aa60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f093c00e040 tx=0x7f093c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f093c00d9f0 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.614+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f093c01a9f0 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.615+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c10b150 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.615+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f093c01b8c0 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.615+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c10b560 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.615+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f093c00f070 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.615+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f093c025ae0 con 0x7f094c070460 2023-12-06T03:20:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.616+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c045ad0 con 0x7f094c070460 2023-12-06T03:20:27.621 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.618+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f093c016070 con 0x7f094c070460 2023-12-06T03:20:27.691 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.689+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f094c0ffbe0 con 0x7f094c070460 2023-12-06T03:20:27.691 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.689+0000 7f093affd700 1 -- 172.21.15.55:0/1440201877 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f093c01bd70 con 0x7f094c070460 2023-12-06T03:20:27.692 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 msgr2=0x7f094c10aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:27.692 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 --2- 172.21.15.55:0/1440201877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c10aa60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f093c00e040 tx=0x7f093c006ee0).stop 2023-12-06T03:20:27.692 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 shutdown_connections 2023-12-06T03:20:27.692 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 --2- 172.21.15.55:0/1440201877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f094c070460 0x7f094c10aa60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:27.692 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 >> 172.21.15.55:0/1440201877 conn(0x7f094c0f63b0 msgr2=0x7f094c0f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:27.692 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 shutdown_connections 2023-12-06T03:20:27.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:27.690+0000 7f0950cb8700 1 -- 172.21.15.55:0/1440201877 wait complete. 2023-12-06T03:20:27.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: audit 2023-12-06T03:20:26.870596+0000 mon.smithi055 (mon.0) 8 : audit [INF] from='client.? 172.21.15.55:0/3402558790' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-06T03:20:27.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: audit 2023-12-06T03:20:26.876206+0000 mon.smithi055 (mon.0) 9 : audit [INF] from='client.? 172.21.15.55:0/3402558790' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-06T03:20:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:Restarting the monitor... 2023-12-06T03:20:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl restart ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055 2023-12-06T03:20:28.426 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 systemd[1]: Stopping Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715... 2023-12-06T03:20:28.426 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: debug 2023-12-06T03:20:28.141+0000 7f46047cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi055 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2023-12-06T03:20:28.426 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111350]: debug 2023-12-06T03:20:28.141+0000 7f46047cb700 -1 mon.smithi055@0(leader) e1 *** Got Signal Terminated *** 2023-12-06T03:20:28.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 podman[111673]: ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055 2023-12-06T03:20:28.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 systemd[1]: ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055.service: Succeeded. 2023-12-06T03:20:28.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 systemd[1]: Stopped Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715. 2023-12-06T03:20:28.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 systemd[1]: Starting Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715... 2023-12-06T03:20:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:Setting mon public_network to 172.21.0.0/20 2023-12-06T03:20:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mon public_network 172.21.0.0/20 2023-12-06T03:20:29.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 bash[111744]: 011f448920619ae0778ca951e6c712fc6c30f827a9342ff701954fd438f83078 2023-12-06T03:20:29.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 systemd[1]: Started Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715. 2023-12-06T03:20:29.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.934+0000 7effdbb9e700 0 set uid:gid to 167:167 (ceph:ceph) 2023-12-06T03:20:29.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.934+0000 7effdbb9e700 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process ceph-mon, pid 7 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.934+0000 7effdbb9e700 0 pidfile_write: ignore empty --pid-file 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.939+0000 7effdbb9e700 0 load: jerasure load: lrc load: isa 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.940+0000 7effdbb9e700 4 rocksdb: RocksDB version: 6.8.1 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.940+0000 7effdbb9e700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.940+0000 7effdbb9e700 4 rocksdb: Compile date May 13 2021 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.940+0000 7effdbb9e700 4 rocksdb: DB SUMMARY 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: CURRENT file: CURRENT 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: IDENTITY file: IDENTITY 2023-12-06T03:20:29.202 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi055/store.db dir, Total Num: 1, files: 000004.sst 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi055/store.db: 000006.log size: 92918 ; 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.error_if_exists: 0 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.create_if_missing: 0 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.paranoid_checks: 1 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.env: 0x55bc590861c0 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.fs: Posix File System 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.info_log: 0x55bc5a3d3960 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_file_opening_threads: 16 2023-12-06T03:20:29.203 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.statistics: (nil) 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.use_fsync: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_log_file_size: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.log_file_time_to_roll: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.keep_log_file_num: 1000 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.recycle_log_file_num: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.allow_fallocate: 1 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.allow_mmap_reads: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.allow_mmap_writes: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.use_direct_reads: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.create_missing_column_families: 0 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.db_log_dir: 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi055/store.db 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.table_cache_numshardbits: 6 2023-12-06T03:20:29.204 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_subcompactions: 1 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_background_flushes: -1 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.WAL_ttl_seconds: 0 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.WAL_size_limit_MB: 0 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.is_fd_close_on_exec: 1 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.advise_random_on_open: 1 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.db_write_buffer_size: 0 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.write_buffer_manager: 0x55bc5b048030 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.use_adaptive_mutex: 0 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.rate_limiter: (nil) 2023-12-06T03:20:29.205 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.wal_recovery_mode: 2 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.enable_thread_tracking: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.enable_pipelined_write: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.unordered_write: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.row_cache: None 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.wal_filter: None 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.allow_ingest_behind: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.preserve_deletes: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.two_write_queues: 0 2023-12-06T03:20:29.206 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.manual_wal_flush: 0 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.atomic_flush: 0 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.persist_stats_to_disk: 0 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.write_dbid_to_manifest: 0 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.log_readahead_size: 0 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_background_jobs: 2 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_background_compactions: -1 2023-12-06T03:20:29.207 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.delayed_write_rate : 16777216 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_total_wal_size: 0 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.stats_dump_period_sec: 600 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.stats_persist_period_sec: 600 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_open_files: -1 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bytes_per_sync: 0 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.wal_bytes_per_sync: 0 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.strict_bytes_per_sync: 0 2023-12-06T03:20:29.209 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_readahead_size: 0 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Compression algorithms supported: 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kZSTD supported: 0 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kXpressCompression supported: 0 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kLZ4HCCompression supported: 1 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kLZ4Compression supported: 1 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kBZip2Compression supported: 0 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kZlibCompression supported: 1 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: kSnappyCompression supported: 1 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Fast CRC32 supported: Supported on x86 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi055/store.db/MANIFEST-000005 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2023-12-06T03:20:29.210 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.merge_operator: 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_filter: None 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_filter_factory: None 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.memtable_factory: SkipListFactory 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.table_factory: BlockBasedTable 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bc5a2ea128) 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cache_index_and_filter_blocks: 1 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: pin_top_level_index_and_filter: 1 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: index_type: 0 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: data_block_index_type: 0 2023-12-06T03:20:29.211 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: index_shortening: 1 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: data_block_hash_table_util_ratio: 0.750000 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: hash_index_allow_collision: 1 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: checksum: 1 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: no_block_cache: 0 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_cache: 0x55bc5a320f10 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_cache_name: BinnedLRUCache 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_cache_options: 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: capacity : 536870912 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: num_shard_bits : 4 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: strict_capacity_limit : 0 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: high_pri_pool_ratio: 0.000 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_cache_compressed: (nil) 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: persistent_cache: (nil) 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_size: 4096 2023-12-06T03:20:29.212 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_size_deviation: 10 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_restart_interval: 16 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: index_block_restart_interval: 1 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: metadata_block_size: 4096 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: partition_filters: 0 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: use_delta_encoding: 1 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: filter_policy: rocksdb.BuiltinBloomFilter 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: whole_key_filtering: 1 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: verify_compression: 0 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: read_amp_bytes_per_bit: 0 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: format_version: 2 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: enable_index_compression: 1 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: block_align: 0 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.213 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.write_buffer_size: 33554432 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_write_buffer_number: 2 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression: NoCompression 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression: Disabled 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.prefix_extractor: nullptr 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.num_levels: 7 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-06T03:20:29.214 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression_opts.window_bits: -14 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression_opts.level: 32767 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression_opts.strategy: 0 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compression_opts.enabled: false 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.target_file_size_base: 67108864 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.target_file_size_multiplier: 1 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-06T03:20:29.215 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.arena_block_size: 4194304 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.disable_auto_compactions: 0 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-06T03:20:29.216 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.table_properties_collectors: 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.inplace_update_support: 0 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.inplace_update_num_locks: 10000 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-06T03:20:29.217 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.memtable_huge_page_size: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.bloom_locality: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.max_successive_merges: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.optimize_filters_for_hits: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.paranoid_file_checks: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.force_consistency_checks: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.report_bg_io_stats: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.ttl: 2592000 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.941+0000 7effdbb9e700 4 rocksdb: Options.periodic_compaction_seconds: 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.942+0000 7effdbb9e700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi055/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.942+0000 7effdbb9e700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.942+0000 7effdbb9e700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701832828943719, "job": 1, "event": "recovery_started", "log_files": [6]} 2023-12-06T03:20:29.218 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.942+0000 7effdbb9e700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.943+0000 7effdbb9e700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.943+0000 7effdbb9e700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701832828944974, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 88698, "table_properties": {"data_size": 86813, "index_size": 362, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 12384, "raw_average_key_size": 49, "raw_value_size": 79954, "raw_average_value_size": 321, "num_data_blocks": 11, "num_entries": 249, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1701832828, "oldest_key_time": 3, "file_creation_time": 0}} 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.943+0000 7effdbb9e700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.944+0000 7effdbb9e700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1701832828945869, "job": 1, "event": "recovery_finished"} 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.945+0000 7effdbb9e700 4 rocksdb: DB pointer 0x55bc5a3e5800 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.946+0000 7effc29b6700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.946+0000 7effc29b6700 4 rocksdb: [db_impl/db_impl.cc:851] 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** DB Stats ** 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Uptime(secs): 0.0 total, 0.0 interval 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-06T03:20:29.219 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** Compaction Stats [default] ** 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: L0 2/0 88.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Sum 2/0 88.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** Compaction Stats [default] ** 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Uptime(secs): 0.0 total, 0.0 interval 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Flush(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:29.220 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(Total Files): cumulative 0, interval 0 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(L0 Files): cumulative 0, interval 0 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(Keys): cumulative 0, interval 0 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Cumulative compaction: 0.00 GB write, 18.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Interval compaction: 0.00 GB write, 18.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** File Read Latency Histogram By Level [default] ** 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** Compaction Stats [default] ** 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: L0 2/0 88.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.221 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Sum 2/0 88.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** Compaction Stats [default] ** 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 99.2 0.00 0.00 1 0.001 0 0 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Uptime(secs): 0.0 total, 0.0 interval 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Flush(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(Total Files): cumulative 0, interval 0 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(L0 Files): cumulative 0, interval 0 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: AddFile(Keys): cumulative 0, interval 0 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Cumulative compaction: 0.00 GB write, 18.36 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-06T03:20:29.222 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ** File Read Latency Histogram By Level [default] ** 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.947+0000 7effdbb9e700 0 starting mon.smithi055 rank 0 at public addrs [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] at bind addrs [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi055 fsid 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 1 mon.smithi055@-1(???) e1 preinit fsid 5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 0 mon.smithi055@-1(???).mds e1 new map 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 0 mon.smithi055@-1(???).mds e1 print_map 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: e1 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: enable_multiple, ever_enabled_multiple: 1,1 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: legacy client fscid: -1 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: No filesystems configured 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: 2023-12-06T03:20:29.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 0 mon.smithi055@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 0 mon.smithi055@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 0 mon.smithi055@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.948+0000 7effdbb9e700 0 mon.smithi055@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:26.870596+0000 mon.smithi055 (mon.0) 8 : audit [INF] from='client.? 172.21.15.55:0/3402558790' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:26.876206+0000 mon.smithi055 (mon.0) 9 : audit [INF] from='client.? 172.21.15.55:0/3402558790' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:20:28.949+0000 7effdbb9e700 1 mon.smithi055@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:28.965908+0000 mon.smithi055 (mon.0) 1 : cluster [INF] mon.smithi055 is new leader, mons smithi055 in quorum (ranks 0) 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:28.965993+0000 mon.smithi055 (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {smithi055=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0]} 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:28.966037+0000 mon.smithi055 (mon.0) 3 : cluster [DBG] fsmap 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:28.966065+0000 mon.smithi055 (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2023-12-06T03:20:29.224 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:28.966627+0000 mon.smithi055 (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.308+0000 7f8818939700 1 Processor -- start 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.308+0000 7f8818939700 1 -- start start 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.308+0000 7f8818939700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f8814100b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.308+0000 7f8818939700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88140fc580 con 0x7f88140703d0 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f8814100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f8814100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58440/0 (socket says 172.21.15.55:58440) 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8812d9d700 1 -- 172.21.15.55:0/613376422 learned_addr learned my addr 172.21.15.55:0/613376422 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:29.311 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8812d9d700 1 -- 172.21.15.55:0/613376422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8814101080 con 0x7f88140703d0 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8812d9d700 1 --2- 172.21.15.55:0/613376422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f8814100b90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f880400b5f0 tx=0x7f8804006ee0).ready entity=mon.0 client_cookie=36f325544e63c397 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8811d9b700 1 -- 172.21.15.55:0/613376422 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880400d600 con 0x7f88140703d0 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8811d9b700 1 -- 172.21.15.55:0/613376422 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f880400dbd0 con 0x7f88140703d0 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.309+0000 7f8811d9b700 1 -- 172.21.15.55:0/613376422 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8804011c00 con 0x7f88140703d0 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 -- 172.21.15.55:0/613376422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 msgr2=0x7f8814100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 --2- 172.21.15.55:0/613376422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f8814100b90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f880400b5f0 tx=0x7f8804006ee0).stop 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 -- 172.21.15.55:0/613376422 shutdown_connections 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 --2- 172.21.15.55:0/613376422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f8814100b90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 -- 172.21.15.55:0/613376422 >> 172.21.15.55:0/613376422 conn(0x7f88140f6210 msgr2=0x7f88140f8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 -- 172.21.15.55:0/613376422 shutdown_connections 2023-12-06T03:20:29.313 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.310+0000 7f8818939700 1 -- 172.21.15.55:0/613376422 wait complete. 2023-12-06T03:20:29.314 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.311+0000 7f8818939700 1 Processor -- start 2023-12-06T03:20:29.314 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.311+0000 7f8818939700 1 -- start start 2023-12-06T03:20:29.314 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.311+0000 7f8818939700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f881410ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:29.314 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.311+0000 7f8818939700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881410cf70 con 0x7f88140703d0 2023-12-06T03:20:29.315 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f881410ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:29.315 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f881410ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58450/0 (socket says 172.21.15.55:58450) 2023-12-06T03:20:29.315 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8812d9d700 1 -- 172.21.15.55:0/320570958 learned_addr learned my addr 172.21.15.55:0/320570958 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8812d9d700 1 -- 172.21.15.55:0/320570958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880400b200 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8812d9d700 1 --2- 172.21.15.55:0/320570958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f881410ca80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f880400e040 tx=0x7f8804006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880400d9f0 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f880401a9f0 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881410d170 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f880401b8c0 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.312+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881410d580 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.313+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f880400f070 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.313+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f8804025b20 con 0x7f88140703d0 2023-12-06T03:20:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.313+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8814045ad0 con 0x7f88140703d0 2023-12-06T03:20:29.317 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.316+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f8804016070 con 0x7f88140703d0 2023-12-06T03:20:29.386 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.384+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f8814101d30 con 0x7f88140703d0 2023-12-06T03:20:29.390 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.389+0000 7f87fb7fe700 1 -- 172.21.15.55:0/320570958 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f880401bdc0 con 0x7f88140703d0 2023-12-06T03:20:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 msgr2=0x7f881410ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 --2- 172.21.15.55:0/320570958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f881410ca80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f880400e040 tx=0x7f8804006ee0).stop 2023-12-06T03:20:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 shutdown_connections 2023-12-06T03:20:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 --2- 172.21.15.55:0/320570958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88140703d0 0x7f881410ca80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 >> 172.21.15.55:0/320570958 conn(0x7f88140f6210 msgr2=0x7f88140f7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 shutdown_connections 2023-12-06T03:20:29.393 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:29.390+0000 7f8818939700 1 -- 172.21.15.55:0/320570958 wait complete. 2023-12-06T03:20:29.758 INFO:teuthology.orchestra.run.smithi055.stderr:Wrote config to /etc/ceph/ceph.conf 2023-12-06T03:20:29.758 INFO:teuthology.orchestra.run.smithi055.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-06T03:20:29.758 INFO:teuthology.orchestra.run.smithi055.stderr:Creating mgr... 2023-12-06T03:20:29.759 INFO:teuthology.orchestra.run.smithi055.stderr:Verifying port 9283 ... 2023-12-06T03:20:29.762 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl daemon-reload 2023-12-06T03:20:29.876 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb 2023-12-06T03:20:29.903 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl reset-failed ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb 2023-12-06T03:20:29.909 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Failed to reset failed state of unit ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb.service: Unit ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb.service not loaded. 2023-12-06T03:20:29.909 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl enable ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb 2023-12-06T03:20:29.916 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: Created symlink /etc/systemd/system/ceph-5555246c-93e6-11ee-95a2-87774f69a715.target.wants/ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb.service → /etc/systemd/system/ceph-5555246c-93e6-11ee-95a2-87774f69a715@.service. 2023-12-06T03:20:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715@mgr.smithi055.qjkkyb 2023-12-06T03:20:30.372 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled firewalld.service 2023-12-06T03:20:30.383 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: disabled 2023-12-06T03:20:30.383 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active firewalld.service 2023-12-06T03:20:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: inactive 2023-12-06T03:20:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:firewalld.service is not enabled 2023-12-06T03:20:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:Not possible to enable service . firewalld.service is not available 2023-12-06T03:20:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:Not possible to open ports <[]>. firewalld.service is not available 2023-12-06T03:20:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled firewalld.service 2023-12-06T03:20:30.401 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: disabled 2023-12-06T03:20:30.402 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active firewalld.service 2023-12-06T03:20:30.411 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: inactive 2023-12-06T03:20:30.412 INFO:teuthology.orchestra.run.smithi055.stderr:firewalld.service is not enabled 2023-12-06T03:20:30.412 INFO:teuthology.orchestra.run.smithi055.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2023-12-06T03:20:30.412 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for mgr to start... 2023-12-06T03:20:30.412 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for mgr... 2023-12-06T03:20:30.413 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2023-12-06T03:20:30.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:30 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:29.390666+0000 mon.smithi055 (mon.0) 6 : audit [INF] from='client.? 172.21.15.55:0/320570958' entity='client.admin' 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.811+0000 7fc7fdba3700 1 Processor -- start 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.811+0000 7fc7fdba3700 1 -- start start 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.812+0000 7fc7fdba3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f80fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.812+0000 7fc7fdba3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80fb490 con 0x7fc7f80fab80 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.812+0000 7fc7f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f80fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.812+0000 7fc7f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f80fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37502/0 (socket says 172.21.15.55:37502) 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.812+0000 7fc7f77fe700 1 -- 172.21.15.55:0/1691717740 learned_addr learned my addr 172.21.15.55:0/1691717740 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.812+0000 7fc7f77fe700 1 -- 172.21.15.55:0/1691717740 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80fb5d0 con 0x7fc7f80fab80 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.813+0000 7fc7f77fe700 1 --2- 172.21.15.55:0/1691717740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f80fafa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc7e800b5f0 tx=0x7fc7e8006ee0).ready entity=mon.0 client_cookie=c1bada169d9fe536 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.813+0000 7fc7f67fc700 1 -- 172.21.15.55:0/1691717740 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7e800d600 con 0x7fc7f80fab80 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.813+0000 7fc7f67fc700 1 -- 172.21.15.55:0/1691717740 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fc7e800dbd0 con 0x7fc7f80fab80 2023-12-06T03:20:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.813+0000 7fc7f67fc700 1 -- 172.21.15.55:0/1691717740 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7e8011c00 con 0x7fc7f80fab80 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.813+0000 7fc7fdba3700 1 -- 172.21.15.55:0/1691717740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 msgr2=0x7fc7f80fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.813+0000 7fc7fdba3700 1 --2- 172.21.15.55:0/1691717740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f80fafa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc7e800b5f0 tx=0x7fc7e8006ee0).stop 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.814+0000 7fc7fdba3700 1 -- 172.21.15.55:0/1691717740 shutdown_connections 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.814+0000 7fc7fdba3700 1 --2- 172.21.15.55:0/1691717740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f80fafa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.814+0000 7fc7fdba3700 1 -- 172.21.15.55:0/1691717740 >> 172.21.15.55:0/1691717740 conn(0x7fc7f80f63c0 msgr2=0x7fc7f80f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.814+0000 7fc7fdba3700 1 -- 172.21.15.55:0/1691717740 shutdown_connections 2023-12-06T03:20:30.817 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.814+0000 7fc7fdba3700 1 -- 172.21.15.55:0/1691717740 wait complete. 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.814+0000 7fc7fdba3700 1 Processor -- start 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7fdba3700 1 -- start start 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7fdba3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7fdba3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80ff290 con 0x7fc7f80fab80 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37518/0 (socket says 172.21.15.55:37518) 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7f77fe700 1 -- 172.21.15.55:0/896099705 learned_addr learned my addr 172.21.15.55:0/896099705 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7f77fe700 1 -- 172.21.15.55:0/896099705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e800b200 con 0x7fc7f80fab80 2023-12-06T03:20:30.819 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7f77fe700 1 --2- 172.21.15.55:0/896099705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f8100e10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc7e800e040 tx=0x7fc7e8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:30.820 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.815+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7e801b410 con 0x7fc7f80fab80 2023-12-06T03:20:30.820 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.816+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f80ff430 con 0x7fc7f80fab80 2023-12-06T03:20:30.820 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.816+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fc7e801b9e0 con 0x7fc7f80fab80 2023-12-06T03:20:30.821 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.816+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7e801a820 con 0x7fc7f80fab80 2023-12-06T03:20:30.821 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.816+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f80ff840 con 0x7fc7f80fab80 2023-12-06T03:20:30.821 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.816+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fc7e800f070 con 0x7fc7f80fab80 2023-12-06T03:20:30.821 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.816+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fc7e8025bd0 con 0x7fc7f80fab80 2023-12-06T03:20:30.821 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.817+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f8045ad0 con 0x7fc7f80fab80 2023-12-06T03:20:30.822 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.819+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fc7e8016070 con 0x7fc7f80fab80 2023-12-06T03:20:30.901 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.892+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc7f80008d0 con 0x7fc7f80fab80 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.893+0000 7fc7dffff700 1 -- 172.21.15.55:0/896099705 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fc7e8025e60 con 0x7fc7f80fab80 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsid": "5555246c-93e6-11ee-95a2-87774f69a715", 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "health": { 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "checks": {}, 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mutes": [] 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum": [ 2023-12-06T03:20:30.902 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 0 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_names": [ 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "smithi055" 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_age": 1, 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "monmap": { 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_mons": 1 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osdmap": { 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_osds": 0, 2023-12-06T03:20:30.903 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgmap": { 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pools": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_objects": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_total": 0 2023-12-06T03:20:30.904 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsmap": { 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "by_rank": [], 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "up:standby": 0 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mgrmap": { 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "available": false, 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modules": [ 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "iostat", 2023-12-06T03:20:30.905 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "restful" 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "servicemap": { 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modified": "2023-12-06T03:20:25.463941+0000", 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "progress_events": {} 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.894+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 msgr2=0x7fc7f8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.894+0000 7fc7fdba3700 1 --2- 172.21.15.55:0/896099705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f8100e10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc7e800e040 tx=0x7fc7e8006ee0).stop 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.894+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 shutdown_connections 2023-12-06T03:20:30.906 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.895+0000 7fc7fdba3700 1 --2- 172.21.15.55:0/896099705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7f80fab80 0x7fc7f8100e10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:30.907 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.895+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 >> 172.21.15.55:0/896099705 conn(0x7fc7f80f63c0 msgr2=0x7fc7f8070090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:30.907 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.895+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 shutdown_connections 2023-12-06T03:20:30.907 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:30.895+0000 7fc7fdba3700 1 -- 172.21.15.55:0/896099705 wait complete. 2023-12-06T03:20:31.319 INFO:teuthology.orchestra.run.smithi055.stderr:mgr not available, waiting (1/15)... 2023-12-06T03:20:31.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:30.894286+0000 mon.smithi055 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.55:0/896099705' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-06T03:20:33.321 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2023-12-06T03:20:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb0956b2700 1 Processor -- start 2023-12-06T03:20:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb0956b2700 1 -- start start 2023-12-06T03:20:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb0956b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb090070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb0956b2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090072940 con 0x7fb090072520 2023-12-06T03:20:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb08effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb090070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb08effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb090070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37534/0 (socket says 172.21.15.55:37534) 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.687+0000 7fb08effd700 1 -- 172.21.15.55:0/3208043528 learned_addr learned my addr 172.21.15.55:0/3208043528 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.688+0000 7fb08effd700 1 -- 172.21.15.55:0/3208043528 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090070fb0 con 0x7fb090072520 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.688+0000 7fb08effd700 1 --2- 172.21.15.55:0/3208043528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb090070ac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fb08000b5f0 tx=0x7fb080006ee0).ready entity=mon.0 client_cookie=110e147ba28d7158 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.688+0000 7fb08dffb700 1 -- 172.21.15.55:0/3208043528 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08000d600 con 0x7fb090072520 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.688+0000 7fb08dffb700 1 -- 172.21.15.55:0/3208043528 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fb08000dbd0 con 0x7fb090072520 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.688+0000 7fb08dffb700 1 -- 172.21.15.55:0/3208043528 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb080011c00 con 0x7fb090072520 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 -- 172.21.15.55:0/3208043528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 msgr2=0x7fb090070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 --2- 172.21.15.55:0/3208043528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb090070ac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fb08000b5f0 tx=0x7fb080006ee0).stop 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 -- 172.21.15.55:0/3208043528 shutdown_connections 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 --2- 172.21.15.55:0/3208043528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb090070ac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 -- 172.21.15.55:0/3208043528 >> 172.21.15.55:0/3208043528 conn(0x7fb0900f74b0 msgr2=0x7fb0900f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 -- 172.21.15.55:0/3208043528 shutdown_connections 2023-12-06T03:20:33.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.689+0000 7fb0956b2700 1 -- 172.21.15.55:0/3208043528 wait complete. 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb0956b2700 1 Processor -- start 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb0956b2700 1 -- start start 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb0956b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb09010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb0956b2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09010d220 con 0x7fb090072520 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb08effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb09010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb08effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb09010cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37538/0 (socket says 172.21.15.55:37538) 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.690+0000 7fb08effd700 1 -- 172.21.15.55:0/121623385 learned_addr learned my addr 172.21.15.55:0/121623385 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb08effd700 1 -- 172.21.15.55:0/121623385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08000b200 con 0x7fb090072520 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb08effd700 1 --2- 172.21.15.55:0/121623385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb09010cd30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fb08000e040 tx=0x7fb080006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08000d9f0 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fb08001a9f0 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09010d420 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb08001b8c0 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09010d830 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fb08000f070 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.691+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fb080025b20 con 0x7fb090072520 2023-12-06T03:20:33.695 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.692+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090045ad0 con 0x7fb090072520 2023-12-06T03:20:33.696 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.695+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fb080016070 con 0x7fb090072520 2023-12-06T03:20:33.771 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.768+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb09010ec80 con 0x7fb090072520 2023-12-06T03:20:33.771 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.769+0000 7fb0777fe700 1 -- 172.21.15.55:0/121623385 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fb08001bdc0 con 0x7fb090072520 2023-12-06T03:20:33.771 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsid": "5555246c-93e6-11ee-95a2-87774f69a715", 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "health": { 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "checks": {}, 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mutes": [] 2023-12-06T03:20:33.772 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-12-06T03:20:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum": [ 2023-12-06T03:20:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 0 2023-12-06T03:20:33.776 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_names": [ 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "smithi055" 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_age": 4, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "monmap": { 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_mons": 1 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osdmap": { 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_osds": 0, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-12-06T03:20:33.777 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgmap": { 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pools": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_objects": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_total": 0 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsmap": { 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "by_rank": [], 2023-12-06T03:20:33.778 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "up:standby": 0 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mgrmap": { 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "available": false, 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modules": [ 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "iostat", 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "restful" 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "servicemap": { 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modified": "2023-12-06T03:20:25.463941+0000", 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "progress_events": {} 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 msgr2=0x7fb09010cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 --2- 172.21.15.55:0/121623385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb09010cd30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fb08000e040 tx=0x7fb080006ee0).stop 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 shutdown_connections 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 --2- 172.21.15.55:0/121623385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb090072520 0x7fb09010cd30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 >> 172.21.15.55:0/121623385 conn(0x7fb0900f74b0 msgr2=0x7fb0900f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 shutdown_connections 2023-12-06T03:20:33.781 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:33.770+0000 7fb0956b2700 1 -- 172.21.15.55:0/121623385 wait complete. 2023-12-06T03:20:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:mgr not available, waiting (2/15)... 2023-12-06T03:20:34.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:33.770152+0000 mon.smithi055 (mon.0) 8 : audit [DBG] from='client.? 172.21.15.55:0/121623385' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-06T03:20:36.129 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2023-12-06T03:20:36.289 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:35.978034+0000 mon.smithi055 (mon.0) 9 : cluster [INF] Activating manager daemon smithi055.qjkkyb 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:35.983329+0000 mon.smithi055 (mon.0) 10 : cluster [DBG] mgrmap e2: smithi055.qjkkyb(active, starting, since 0.00543007s) 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:35.984266+0000 mon.smithi055 (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:35.984427+0000 mon.smithi055 (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:35.984632+0000 mon.smithi055 (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:35.984841+0000 mon.smithi055 (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata", "id": "smithi055"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:35.985115+0000 mon.smithi055 (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mgr metadata", "who": "smithi055.qjkkyb", "id": "smithi055.qjkkyb"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:35.998986+0000 mon.smithi055 (mon.0) 16 : cluster [INF] Manager daemon smithi055.qjkkyb is now available 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:36.006378+0000 mon.smithi055 (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:36.013446+0000 mon.smithi055 (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:36.014904+0000 mon.smithi055 (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:20:36.290 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:36.020452+0000 mon.smithi055 (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:36.291 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:36.023300+0000 mon.smithi055 (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.55:0/3491320080' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:36.478 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.476+0000 7f119672f700 1 Processor -- start 2023-12-06T03:20:36.479 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.476+0000 7f119672f700 1 -- start start 2023-12-06T03:20:36.479 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.476+0000 7f119672f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:36.479 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.476+0000 7f119672f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11900fedd0 con 0x7f1190070430 2023-12-06T03:20:36.479 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.477+0000 7f118ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:36.479 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.477+0000 7f118ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37580/0 (socket says 172.21.15.55:37580) 2023-12-06T03:20:36.479 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.477+0000 7f118ffff700 1 -- 172.21.15.55:0/2600585847 learned_addr learned my addr 172.21.15.55:0/2600585847 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.477+0000 7f118ffff700 1 -- 172.21.15.55:0/2600585847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900fef10 con 0x7f1190070430 2023-12-06T03:20:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.477+0000 7f118ffff700 1 --2- 172.21.15.55:0/2600585847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fe8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f118000b5f0 tx=0x7f1180006ee0).ready entity=mon.0 client_cookie=eabef57d92d5f1b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.477+0000 7f118effd700 1 -- 172.21.15.55:0/2600585847 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f118000d600 con 0x7f1190070430 2023-12-06T03:20:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f118effd700 1 -- 172.21.15.55:0/2600585847 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f118000dbd0 con 0x7f1190070430 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f118effd700 1 -- 172.21.15.55:0/2600585847 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1180011c00 con 0x7f1190070430 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 -- 172.21.15.55:0/2600585847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 msgr2=0x7f11900fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 --2- 172.21.15.55:0/2600585847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fe8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f118000b5f0 tx=0x7f1180006ee0).stop 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 -- 172.21.15.55:0/2600585847 shutdown_connections 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 --2- 172.21.15.55:0/2600585847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fe8e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 -- 172.21.15.55:0/2600585847 >> 172.21.15.55:0/2600585847 conn(0x7f11900f63c0 msgr2=0x7f11900f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 -- 172.21.15.55:0/2600585847 shutdown_connections 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.478+0000 7f119672f700 1 -- 172.21.15.55:0/2600585847 wait complete. 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.479+0000 7f119672f700 1 Processor -- start 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.479+0000 7f119672f700 1 -- start start 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.479+0000 7f119672f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.479+0000 7f119672f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11900fa7e0 con 0x7f1190070430 2023-12-06T03:20:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.479+0000 7f118ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37594/0 (socket says 172.21.15.55:37594) 2023-12-06T03:20:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118ffff700 1 -- 172.21.15.55:0/1071458128 learned_addr learned my addr 172.21.15.55:0/1071458128 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118ffff700 1 -- 172.21.15.55:0/1071458128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118000b200 con 0x7f1190070430 2023-12-06T03:20:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118ffff700 1 --2- 172.21.15.55:0/1071458128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fc360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f118000e040 tx=0x7f1180006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f118000d9f0 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f118001a9f0 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f118001b8c0 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11900fa980 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.480+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11900fad90 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.481+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 2) v1 ==== 35959+0+0 (secure 0 0 0) 0x7f118000f070 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.481+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f1180041090 con 0x7f1190070430 2023-12-06T03:20:36.484 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.481+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1190045ad0 con 0x7f1190070430 2023-12-06T03:20:36.485 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.484+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f1180016080 con 0x7f1190070430 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.556+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f11900ffb90 con 0x7f1190070430 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.557+0000 7f118cff9700 1 -- 172.21.15.55:0/1071458128 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7f1180022720 con 0x7f1190070430 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsid": "5555246c-93e6-11ee-95a2-87774f69a715", 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "health": { 2023-12-06T03:20:36.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "checks": {}, 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mutes": [] 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum": [ 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 0 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_names": [ 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "smithi055" 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_age": 7, 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "monmap": { 2023-12-06T03:20:36.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_mons": 1 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osdmap": { 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_osds": 0, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgmap": { 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pools": 0, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_objects": 0, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_total": 0 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsmap": { 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "by_rank": [], 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "up:standby": 0 2023-12-06T03:20:36.568 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mgrmap": { 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "available": false, 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modules": [ 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "iostat", 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "restful" 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "servicemap": { 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modified": "2023-12-06T03:20:25.463941+0000", 2023-12-06T03:20:36.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "progress_events": {} 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.557+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 msgr2=0x7f11900fc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.557+0000 7f119672f700 1 --2- 172.21.15.55:0/1071458128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fc360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f118000e040 tx=0x7f1180006ee0).stop 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.558+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 shutdown_connections 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.558+0000 7f119672f700 1 --2- 172.21.15.55:0/1071458128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1190070430 0x7f11900fc360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.558+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 >> 172.21.15.55:0/1071458128 conn(0x7f11900f63c0 msgr2=0x7f11900f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.558+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 shutdown_connections 2023-12-06T03:20:36.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:36.558+0000 7f119672f700 1 -- 172.21.15.55:0/1071458128 wait complete. 2023-12-06T03:20:36.910 INFO:teuthology.orchestra.run.smithi055.stderr:mgr not available, waiting (3/15)... 2023-12-06T03:20:37.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:36.558059+0000 mon.smithi055 (mon.0) 22 : audit [DBG] from='client.? 172.21.15.55:0/1071458128' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-06T03:20:37.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:36.986249+0000 mon.smithi055 (mon.0) 23 : cluster [DBG] mgrmap e3: smithi055.qjkkyb(active, since 1.00834s) 2023-12-06T03:20:38.913 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2023-12-06T03:20:39.315 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.313+0000 7fb5295f6700 1 Processor -- start 2023-12-06T03:20:39.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.313+0000 7fb5295f6700 1 -- start start 2023-12-06T03:20:39.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.313+0000 7fb5295f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb5240fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:39.316 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.313+0000 7fb5295f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240fc410 con 0x7fb5240fbb00 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb523fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb5240fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb523fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb5240fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37604/0 (socket says 172.21.15.55:37604) 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb523fff700 1 -- 172.21.15.55:0/2366238112 learned_addr learned my addr 172.21.15.55:0/2366238112 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb523fff700 1 -- 172.21.15.55:0/2366238112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240fc550 con 0x7fb5240fbb00 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb523fff700 1 --2- 172.21.15.55:0/2366238112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb5240fbf20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb51800b5f0 tx=0x7fb518006ee0).ready entity=mon.0 client_cookie=d8950744ddcfd798 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb522ffd700 1 -- 172.21.15.55:0/2366238112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb51800d600 con 0x7fb5240fbb00 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb522ffd700 1 -- 172.21.15.55:0/2366238112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fb51800dbd0 con 0x7fb5240fbb00 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.314+0000 7fb522ffd700 1 -- 172.21.15.55:0/2366238112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb518011c00 con 0x7fb5240fbb00 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 -- 172.21.15.55:0/2366238112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 msgr2=0x7fb5240fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 --2- 172.21.15.55:0/2366238112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb5240fbf20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb51800b5f0 tx=0x7fb518006ee0).stop 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 -- 172.21.15.55:0/2366238112 shutdown_connections 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 --2- 172.21.15.55:0/2366238112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb5240fbf20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:39.318 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 -- 172.21.15.55:0/2366238112 >> 172.21.15.55:0/2366238112 conn(0x7fb5240f7380 msgr2=0x7fb5240f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 -- 172.21.15.55:0/2366238112 shutdown_connections 2023-12-06T03:20:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.315+0000 7fb5295f6700 1 -- 172.21.15.55:0/2366238112 wait complete. 2023-12-06T03:20:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.316+0000 7fb5295f6700 1 Processor -- start 2023-12-06T03:20:39.319 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.316+0000 7fb5295f6700 1 -- start start 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb5295f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb52410a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb5295f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52410adc0 con 0x7fb5240fbb00 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb523fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb52410a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb523fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb52410a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37616/0 (socket says 172.21.15.55:37616) 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb523fff700 1 -- 172.21.15.55:0/4215691955 learned_addr learned my addr 172.21.15.55:0/4215691955 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb523fff700 1 -- 172.21.15.55:0/4215691955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51800b200 con 0x7fb5240fbb00 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb523fff700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb52410a8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb518000c00 tx=0x7fb518006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:39.321 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb518011690 con 0x7fb5240fbb00 2023-12-06T03:20:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fb51800d9f0 con 0x7fb5240fbb00 2023-12-06T03:20:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52410afc0 con 0x7fb5240fbb00 2023-12-06T03:20:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb51801b710 con 0x7fb5240fbb00 2023-12-06T03:20:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.317+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52410b3d0 con 0x7fb5240fbb00 2023-12-06T03:20:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.318+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7fb51800f070 con 0x7fb5240fbb00 2023-12-06T03:20:39.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.318+0000 7fb520ff9700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fb51002fd50 0x7fb510032210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:39.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.318+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fb5180410b0 con 0x7fb5240fbb00 2023-12-06T03:20:39.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.318+0000 7fb5237fe700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fb51002fd50 0x7fb510032210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:39.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.319+0000 7fb5237fe700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fb51002fd50 0x7fb510032210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb5140096a0 tx=0x7fb514006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:39.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.319+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524059100 con 0x7fb5240fbb00 2023-12-06T03:20:39.325 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.324+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb518016070 con 0x7fb5240fbb00 2023-12-06T03:20:39.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:38.033839+0000 mon.smithi055 (mon.0) 24 : cluster [DBG] mgrmap e4: smithi055.qjkkyb(active, since 2s) 2023-12-06T03:20:39.560 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.552+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb5241046e0 con 0x7fb5240fbb00 2023-12-06T03:20:39.560 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.552+0000 7fb520ff9700 1 -- 172.21.15.55:0/4215691955 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fb5180226d0 con 0x7fb5240fbb00 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsid": "5555246c-93e6-11ee-95a2-87774f69a715", 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "health": { 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "checks": {}, 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mutes": [] 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "election_epoch": 5, 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum": [ 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 0 2023-12-06T03:20:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_names": [ 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "smithi055" 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "quorum_age": 10, 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "monmap": { 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_mons": 1 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osdmap": { 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_osds": 0, 2023-12-06T03:20:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_up_osds": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_up_since": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_in_osds": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "osd_in_since": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgmap": { 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "pgs_by_state": [], 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pgs": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_pools": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_objects": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "data_bytes": 0, 2023-12-06T03:20:39.563 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_used": 0, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_avail": 0, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "bytes_total": 0 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "fsmap": { 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "by_rank": [], 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "up:standby": 0 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mgrmap": { 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "available": true, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_standbys": 0, 2023-12-06T03:20:39.564 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modules": [ 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "iostat", 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "restful" 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ], 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "servicemap": { 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 1, 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "modified": "2023-12-06T03:20:25.463941+0000", 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "services": {} 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: }, 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "progress_events": {} 2023-12-06T03:20:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fb51002fd50 msgr2=0x7fb510032210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fb51002fd50 0x7fb510032210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fb5140096a0 tx=0x7fb514006b40).stop 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 msgr2=0x7fb52410a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb52410a8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb518000c00 tx=0x7fb518006ee0).stop 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 shutdown_connections 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fb51002fd50 0x7fb510032210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 --2- 172.21.15.55:0/4215691955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5240fbb00 0x7fb52410a8d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 >> 172.21.15.55:0/4215691955 conn(0x7fb5240f7380 msgr2=0x7fb5240f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 shutdown_connections 2023-12-06T03:20:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:39.554+0000 7fb5295f6700 1 -- 172.21.15.55:0/4215691955 wait complete. 2023-12-06T03:20:39.901 INFO:teuthology.orchestra.run.smithi055.stderr:mgr is available 2023-12-06T03:20:39.902 INFO:teuthology.orchestra.run.smithi055.stderr:Enabling cephadm module... 2023-12-06T03:20:39.902 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable cephadm 2023-12-06T03:20:40.263 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.261+0000 7f517999f700 1 Processor -- start 2023-12-06T03:20:40.263 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.261+0000 7f517999f700 1 -- start start 2023-12-06T03:20:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.261+0000 7f517999f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f51740fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f517999f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740fec20 con 0x7f5174070430 2023-12-06T03:20:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f517899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f51740fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f517899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f51740fe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53600/0 (socket says 172.21.15.55:53600) 2023-12-06T03:20:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f517899d700 1 -- 172.21.15.55:0/29162859 learned_addr learned my addr 172.21.15.55:0/29162859 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:40.264 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f517899d700 1 -- 172.21.15.55:0/29162859 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740fed60 con 0x7f5174070430 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f517899d700 1 --2- 172.21.15.55:0/29162859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f51740fe730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f516800b5f0 tx=0x7f5168006ee0).ready entity=mon.0 client_cookie=ebeb9dec43ef2e59 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f51737fe700 1 -- 172.21.15.55:0/29162859 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f516800d600 con 0x7f5174070430 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f51737fe700 1 -- 172.21.15.55:0/29162859 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f516800dbd0 con 0x7f5174070430 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.262+0000 7f51737fe700 1 -- 172.21.15.55:0/29162859 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5168011c00 con 0x7f5174070430 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 -- 172.21.15.55:0/29162859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 msgr2=0x7f51740fe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 --2- 172.21.15.55:0/29162859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f51740fe730 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f516800b5f0 tx=0x7f5168006ee0).stop 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 -- 172.21.15.55:0/29162859 shutdown_connections 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 --2- 172.21.15.55:0/29162859 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f51740fe730 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 -- 172.21.15.55:0/29162859 >> 172.21.15.55:0/29162859 conn(0x7f51740f6210 msgr2=0x7f51740f8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 -- 172.21.15.55:0/29162859 shutdown_connections 2023-12-06T03:20:40.267 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.263+0000 7f517999f700 1 -- 172.21.15.55:0/29162859 wait complete. 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517999f700 1 Processor -- start 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517999f700 1 -- start start 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517999f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f517410ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517999f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517410cf20 con 0x7f5174070430 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f517410ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f517410ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53616/0 (socket says 172.21.15.55:53616) 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.264+0000 7f517899d700 1 -- 172.21.15.55:0/1535918967 learned_addr learned my addr 172.21.15.55:0/1535918967 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f517899d700 1 -- 172.21.15.55:0/1535918967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516800b200 con 0x7f5174070430 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f517899d700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f517410ca30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f516800b190 tx=0x7f5168006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f516801b410 con 0x7f5174070430 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f516801b9e0 con 0x7f5174070430 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517410d120 con 0x7f5174070430 2023-12-06T03:20:40.268 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f516801a880 con 0x7f5174070430 2023-12-06T03:20:40.269 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.265+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517410d530 con 0x7f5174070430 2023-12-06T03:20:40.269 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.266+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7f516800f070 con 0x7f5174070430 2023-12-06T03:20:40.269 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.266+0000 7f51717fa700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f515802fd00 0x7f51580321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:40.270 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.266+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f5168042130 con 0x7f5174070430 2023-12-06T03:20:40.270 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.266+0000 7f5173fff700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f515802fd00 0x7f51580321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:40.270 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.266+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5174059100 con 0x7f5174070430 2023-12-06T03:20:40.270 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.266+0000 7f5173fff700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f515802fd00 0x7f51580321c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f51600096a0 tx=0x7f5160006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:40.272 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.271+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5168016070 con 0x7f5174070430 2023-12-06T03:20:40.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:40 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:39.553451+0000 mon.smithi055 (mon.0) 25 : audit [DBG] from='client.? 172.21.15.55:0/4215691955' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-06T03:20:40.481 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:40.479+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f5174045ad0 con 0x7f5174070430 2023-12-06T03:20:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.041+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f51680257e0 con 0x7f5174070430 2023-12-06T03:20:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.042+0000 7f51717fa700 1 -- 172.21.15.55:0/1535918967 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f51680105f0 con 0x7f5174070430 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f515802fd00 msgr2=0x7f51580321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f515802fd00 0x7f51580321c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f51600096a0 tx=0x7f5160006b40).stop 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 msgr2=0x7f517410ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f517410ca30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f516800b190 tx=0x7f5168006ee0).stop 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 shutdown_connections 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f515802fd00 0x7f51580321c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 --2- 172.21.15.55:0/1535918967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5174070430 0x7f517410ca30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.045+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 >> 172.21.15.55:0/1535918967 conn(0x7f51740f6210 msgr2=0x7f51740f7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.046+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 shutdown_connections 2023-12-06T03:20:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:41.046+0000 7f517999f700 1 -- 172.21.15.55:0/1535918967 wait complete. 2023-12-06T03:20:41.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:40.481152+0000 mon.smithi055 (mon.0) 26 : audit [INF] from='client.? 172.21.15.55:0/1535918967' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-06T03:20:42.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:42 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:41.043835+0000 mon.smithi055 (mon.0) 27 : audit [INF] from='client.? 172.21.15.55:0/1535918967' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-06T03:20:42.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:42 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:41.043922+0000 mon.smithi055 (mon.0) 28 : cluster [DBG] mgrmap e5: smithi055.qjkkyb(active, since 5s) 2023-12-06T03:20:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2023-12-06T03:20:43.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.677+0000 7f41edb1e700 1 Processor -- start 2023-12-06T03:20:43.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.677+0000 7f41edb1e700 1 -- start start 2023-12-06T03:20:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.677+0000 7f41edb1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e80fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.677+0000 7f41edb1e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80fb490 con 0x7f41e80fab80 2023-12-06T03:20:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.678+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e80fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.678+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e80fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53636/0 (socket says 172.21.15.55:53636) 2023-12-06T03:20:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.678+0000 7f41e77fe700 1 -- 172.21.15.55:0/1752087236 learned_addr learned my addr 172.21.15.55:0/1752087236 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.678+0000 7f41e77fe700 1 -- 172.21.15.55:0/1752087236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80fb5d0 con 0x7f41e80fab80 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.678+0000 7f41e77fe700 1 --2- 172.21.15.55:0/1752087236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e80fafa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f41d800b5f0 tx=0x7f41d8006ee0).ready entity=mon.0 client_cookie=d56f48fd2487fb8d server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.678+0000 7f41e67fc700 1 -- 172.21.15.55:0/1752087236 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d800d600 con 0x7f41e80fab80 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41e67fc700 1 -- 172.21.15.55:0/1752087236 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f41d800dbd0 con 0x7f41e80fab80 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41e67fc700 1 -- 172.21.15.55:0/1752087236 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d8011c00 con 0x7f41e80fab80 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 -- 172.21.15.55:0/1752087236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 msgr2=0x7f41e80fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 --2- 172.21.15.55:0/1752087236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e80fafa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f41d800b5f0 tx=0x7f41d8006ee0).stop 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 -- 172.21.15.55:0/1752087236 shutdown_connections 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 --2- 172.21.15.55:0/1752087236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e80fafa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 -- 172.21.15.55:0/1752087236 >> 172.21.15.55:0/1752087236 conn(0x7f41e80f63c0 msgr2=0x7f41e80f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 -- 172.21.15.55:0/1752087236 shutdown_connections 2023-12-06T03:20:43.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.679+0000 7f41edb1e700 1 -- 172.21.15.55:0/1752087236 wait complete. 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.680+0000 7f41edb1e700 1 Processor -- start 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.680+0000 7f41edb1e700 1 -- start start 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41edb1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e810d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41edb1e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8070470 con 0x7f41e80fab80 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e810d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e810d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53640/0 (socket says 172.21.15.55:53640) 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41e77fe700 1 -- 172.21.15.55:0/2490923571 learned_addr learned my addr 172.21.15.55:0/2490923571 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41e77fe700 1 -- 172.21.15.55:0/2490923571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d800b200 con 0x7f41e80fab80 2023-12-06T03:20:43.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.681+0000 7f41e77fe700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e810d720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f41d800e040 tx=0x7f41d8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:43.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.682+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d801b410 con 0x7f41e80fab80 2023-12-06T03:20:43.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.682+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8070670 con 0x7f41e80fab80 2023-12-06T03:20:43.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.682+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f41d801b9e0 con 0x7f41e80fab80 2023-12-06T03:20:43.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.682+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e810ddd0 con 0x7f41e80fab80 2023-12-06T03:20:43.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.682+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d801a820 con 0x7f41e80fab80 2023-12-06T03:20:43.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.683+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f41d800f070 con 0x7f41e80fab80 2023-12-06T03:20:43.686 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.683+0000 7f41cffff700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 0x7f41d4032210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:43.686 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.683+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f41d80413f0 con 0x7f41e80fab80 2023-12-06T03:20:43.686 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.683+0000 7f41e6ffd700 1 -- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 msgr2=0x7f41d4032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/962833525 2023-12-06T03:20:43.686 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.683+0000 7f41e6ffd700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 0x7f41d4032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-12-06T03:20:43.686 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.683+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e8045ad0 con 0x7f41e80fab80 2023-12-06T03:20:43.693 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.691+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f41d8016070 con 0x7f41e80fab80 2023-12-06T03:20:43.885 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.884+0000 7f41e6ffd700 1 -- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 msgr2=0x7f41d4032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/962833525 2023-12-06T03:20:43.886 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.884+0000 7f41e6ffd700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 0x7f41d4032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-12-06T03:20:43.918 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.916+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f41e80fc890 con 0x7f41e80fab80 2023-12-06T03:20:43.918 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.917+0000 7f41cffff700 1 -- 172.21.15.55:0/2490923571 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f41d8025460 con 0x7f41e80fab80 2023-12-06T03:20:43.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:20:43.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 5, 2023-12-06T03:20:43.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "available": true, 2023-12-06T03:20:43.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "active_name": "smithi055.qjkkyb", 2023-12-06T03:20:43.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_standby": 0 2023-12-06T03:20:43.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:20:43.921 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 msgr2=0x7f41d4032210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 0x7f41d4032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 msgr2=0x7f41e810d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e810d720 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f41d800e040 tx=0x7f41d8006ee0).stop 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 shutdown_connections 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7f41d402fd50 0x7f41d4032210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 --2- 172.21.15.55:0/2490923571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e80fab80 0x7f41e810d720 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 >> 172.21.15.55:0/2490923571 conn(0x7f41e80f63c0 msgr2=0x7f41e80f7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 shutdown_connections 2023-12-06T03:20:43.922 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:43.919+0000 7f41edb1e700 1 -- 172.21.15.55:0/2490923571 wait complete. 2023-12-06T03:20:44.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:43.917998+0000 mon.smithi055 (mon.0) 29 : audit [DBG] from='client.? 172.21.15.55:0/2490923571' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-06T03:20:44.275 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for the mgr to restart... 2023-12-06T03:20:44.275 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for mgr epoch 5... 2023-12-06T03:20:44.276 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.621+0000 7fd566041700 1 Processor -- start 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.621+0000 7fd566041700 1 -- start start 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.621+0000 7fd566041700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd560100d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.621+0000 7fd566041700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600fc730 con 0x7fd5600703d0 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd560100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd560100d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53642/0 (socket says 172.21.15.55:53642) 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55f7fe700 1 -- 172.21.15.55:0/1980423910 learned_addr learned my addr 172.21.15.55:0/1980423910 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55f7fe700 1 -- 172.21.15.55:0/1980423910 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560101230 con 0x7fd5600703d0 2023-12-06T03:20:44.624 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55f7fe700 1 --2- 172.21.15.55:0/1980423910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd560100d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd55000b5f0 tx=0x7fd550006ee0).ready entity=mon.0 client_cookie=45d1c4a206430b0b server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:44.626 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55e7fc700 1 -- 172.21.15.55:0/1980423910 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd55000d600 con 0x7fd5600703d0 2023-12-06T03:20:44.626 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55e7fc700 1 -- 172.21.15.55:0/1980423910 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fd55000dbd0 con 0x7fd5600703d0 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.622+0000 7fd55e7fc700 1 -- 172.21.15.55:0/1980423910 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd550011c00 con 0x7fd5600703d0 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 -- 172.21.15.55:0/1980423910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 msgr2=0x7fd560100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 --2- 172.21.15.55:0/1980423910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd560100d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd55000b5f0 tx=0x7fd550006ee0).stop 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 -- 172.21.15.55:0/1980423910 shutdown_connections 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 --2- 172.21.15.55:0/1980423910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd560100d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 -- 172.21.15.55:0/1980423910 >> 172.21.15.55:0/1980423910 conn(0x7fd5600f63c0 msgr2=0x7fd5600f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 -- 172.21.15.55:0/1980423910 shutdown_connections 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.623+0000 7fd566041700 1 -- 172.21.15.55:0/1980423910 wait complete. 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.624+0000 7fd566041700 1 Processor -- start 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd566041700 1 -- start start 2023-12-06T03:20:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd566041700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd56010cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:44.628 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd566041700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56010d0a0 con 0x7fd5600703d0 2023-12-06T03:20:44.628 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd56010cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:44.628 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd56010cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53652/0 (socket says 172.21.15.55:53652) 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd55f7fe700 1 -- 172.21.15.55:0/615552447 learned_addr learned my addr 172.21.15.55:0/615552447 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd55f7fe700 1 -- 172.21.15.55:0/615552447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55000b200 con 0x7fd5600703d0 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.625+0000 7fd55f7fe700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd56010cbb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd55000b190 tx=0x7fd550006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.626+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd55001b410 con 0x7fd5600703d0 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.626+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fd55001b9e0 con 0x7fd5600703d0 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.626+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56010d2a0 con 0x7fd5600703d0 2023-12-06T03:20:44.629 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.626+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd55001a880 con 0x7fd5600703d0 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.626+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56010d6b0 con 0x7fd5600703d0 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.627+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fd55000f070 con 0x7fd5600703d0 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.627+0000 7fd547fff700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.627+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 --> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd560059100 con 0x7fd54c02fd50 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.627+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fd550041210 con 0x7fd5600703d0 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.627+0000 7fd55effd700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 msgr2=0x7fd54c032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/962833525 2023-12-06T03:20:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.627+0000 7fd55effd700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-12-06T03:20:44.830 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.828+0000 7fd55effd700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 msgr2=0x7fd54c032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/962833525 2023-12-06T03:20:44.830 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:44.828+0000 7fd55effd700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-12-06T03:20:45.230 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:45.229+0000 7fd55effd700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 msgr2=0x7fd54c032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/962833525 2023-12-06T03:20:45.231 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:45.229+0000 7fd55effd700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-12-06T03:20:46.031 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:46.030+0000 7fd55effd700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 msgr2=0x7fd54c032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/962833525 2023-12-06T03:20:46.032 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:46.030+0000 7fd55effd700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-12-06T03:20:46.737 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:46.735+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mgrmap(e 6) v1 ==== 35970+0+0 (secure 0 0 0) 0x7fd550024ca0 con 0x7fd5600703d0 2023-12-06T03:20:46.737 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:46.736+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 msgr2=0x7fd54c032210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-06T03:20:46.737 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:46.736+0000 7fd547fff700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:20:47.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:46.684345+0000 mon.smithi055 (mon.0) 30 : cluster [INF] Active manager daemon smithi055.qjkkyb restarted 2023-12-06T03:20:47.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:46.684878+0000 mon.smithi055 (mon.0) 31 : cluster [INF] Activating manager daemon smithi055.qjkkyb 2023-12-06T03:20:47.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:46.688889+0000 mon.smithi055 (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2023-12-06T03:20:47.747 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.745+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7fd550025460 con 0x7fd5600703d0 2023-12-06T03:20:47.748 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.745+0000 7fd547fff700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fd54c033670 0x7fd54c035a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:47.748 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.745+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd560059100 con 0x7fd54c033670 2023-12-06T03:20:47.748 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.746+0000 7fd55effd700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fd54c033670 0x7fd54c035a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:47.748 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.747+0000 7fd55effd700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fd54c033670 0x7fd54c035a60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fd5540036d0 tx=0x7fd554006ca0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:47.749 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.747+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fd560059100 con 0x7fd54c033670 2023-12-06T03:20:47.755 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.754+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd560045ad0 con 0x7fd54c033670 2023-12-06T03:20:47.755 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.754+0000 7fd547fff700 1 -- 172.21.15.55:0/615552447 <== mgr.14116 v2:172.21.15.55:6800/3025569119 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd560045ad0 con 0x7fd54c033670 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "initialized": true 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.754+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fd54c033670 msgr2=0x7fd54c035a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.754+0000 7fd566041700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fd54c033670 0x7fd54c035a60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7fd5540036d0 tx=0x7fd554006ca0).stop 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.754+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 msgr2=0x7fd56010cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:47.757 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.754+0000 7fd566041700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd56010cbb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd55000b190 tx=0x7fd550006ee0).stop 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 shutdown_connections 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fd54c033670 0x7fd54c035a60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5600703d0 0x7fd56010cbb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 --2- 172.21.15.55:0/615552447 >> [v2:172.21.15.55:6800/962833525,v1:172.21.15.55:6801/962833525] conn(0x7fd54c02fd50 0x7fd54c032210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 >> 172.21.15.55:0/615552447 conn(0x7fd5600f63c0 msgr2=0x7fd5600f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 shutdown_connections 2023-12-06T03:20:47.758 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:47.755+0000 7fd566041700 1 -- 172.21.15.55:0/615552447 wait complete. 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:46.738017+0000 mon.smithi055 (mon.0) 33 : cluster [DBG] mgrmap e6: smithi055.qjkkyb(active, starting, since 0.0532203s) 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.740521+0000 mon.smithi055 (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata", "id": "smithi055"}]: dispatch 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.741288+0000 mon.smithi055 (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mgr metadata", "who": "smithi055.qjkkyb", "id": "smithi055.qjkkyb"}]: dispatch 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.742292+0000 mon.smithi055 (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.742463+0000 mon.smithi055 (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.742612+0000 mon.smithi055 (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-06T03:20:48.029 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:46.744830+0000 mon.smithi055 (mon.0) 39 : cluster [INF] Manager daemon smithi055.qjkkyb is now available 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.761212+0000 mon.smithi055 (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.764323+0000 mon.smithi055 (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.767270+0000 mon.smithi055 (mon.0) 42 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.871413+0000 mon.smithi055 (mon.0) 43 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.872190+0000 mon.smithi055 (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.873557+0000 mon.smithi055 (mon.0) 45 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.875155+0000 mon.smithi055 (mon.0) 46 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.887039+0000 mon.smithi055 (mon.0) 47 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:20:48.030 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:46.888349+0000 mon.smithi055 (mon.0) 48 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:20:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:mgr epoch 5 is available 2023-12-06T03:20:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:Setting orchestrator backend to cephadm... 2023-12-06T03:20:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch set backend cephadm 2023-12-06T03:20:48.498 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b6a694700 1 Processor -- start 2023-12-06T03:20:48.498 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b6a694700 1 -- start start 2023-12-06T03:20:48.498 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b6a694700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b64100d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:48.499 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b6a694700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640fc770 con 0x7f8b64070390 2023-12-06T03:20:48.499 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b64100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:48.499 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b64100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53706/0 (socket says 172.21.15.55:53706) 2023-12-06T03:20:48.499 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.495+0000 7f8b63fff700 1 -- 172.21.15.55:0/3990682561 learned_addr learned my addr 172.21.15.55:0/3990682561 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:48.499 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.496+0000 7f8b63fff700 1 -- 172.21.15.55:0/3990682561 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b64101270 con 0x7f8b64070390 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.496+0000 7f8b63fff700 1 --2- 172.21.15.55:0/3990682561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b64100d80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f8b5400b5f0 tx=0x7f8b54006ee0).ready entity=mon.0 client_cookie=38756b4667d6d317 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.496+0000 7f8b62ffd700 1 -- 172.21.15.55:0/3990682561 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b5400d600 con 0x7f8b64070390 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.496+0000 7f8b62ffd700 1 -- 172.21.15.55:0/3990682561 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8b5400dbd0 con 0x7f8b64070390 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.496+0000 7f8b62ffd700 1 -- 172.21.15.55:0/3990682561 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b54011c00 con 0x7f8b64070390 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 -- 172.21.15.55:0/3990682561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 msgr2=0x7f8b64100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 --2- 172.21.15.55:0/3990682561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b64100d80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f8b5400b5f0 tx=0x7f8b54006ee0).stop 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 -- 172.21.15.55:0/3990682561 shutdown_connections 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 --2- 172.21.15.55:0/3990682561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b64100d80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 -- 172.21.15.55:0/3990682561 >> 172.21.15.55:0/3990682561 conn(0x7f8b640f63c0 msgr2=0x7f8b640f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 -- 172.21.15.55:0/3990682561 shutdown_connections 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.497+0000 7f8b6a694700 1 -- 172.21.15.55:0/3990682561 wait complete. 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.498+0000 7f8b6a694700 1 Processor -- start 2023-12-06T03:20:48.502 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.498+0000 7f8b6a694700 1 -- start start 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.498+0000 7f8b6a694700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b6410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b6a694700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6410d0f0 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b6410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b6410cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53718/0 (socket says 172.21.15.55:53718) 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b63fff700 1 -- 172.21.15.55:0/1608193406 learned_addr learned my addr 172.21.15.55:0/1608193406 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b63fff700 1 -- 172.21.15.55:0/1608193406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5400b200 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b63fff700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b6410cc00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8b5400b190 tx=0x7f8b54006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b5401b410 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8b5401b9e0 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6410d2f0 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b5401a880 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.499+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6410d700 con 0x7f8b64070390 2023-12-06T03:20:48.503 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.500+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7f8b5400f070 con 0x7f8b64070390 2023-12-06T03:20:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.501+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b64045ad0 con 0x7f8b64070390 2023-12-06T03:20:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.501+0000 7f8b60ff9700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8b4402fcf0 0x7f8b440321b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:48.504 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.501+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f8b540410f0 con 0x7f8b64070390 2023-12-06T03:20:48.505 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.501+0000 7f8b637fe700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8b4402fcf0 0x7f8b440321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:48.505 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.501+0000 7f8b637fe700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8b4402fcf0 0x7f8b440321b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f8b4c0096a0 tx=0x7f8b4c006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:48.507 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.506+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8b54022310 con 0x7f8b64070390 2023-12-06T03:20:48.705 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.703+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b6410e040 con 0x7f8b4402fcf0 2023-12-06T03:20:48.712 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.711+0000 7f8b60ff9700 1 -- 172.21.15.55:0/1608193406 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b6410e040 con 0x7f8b4402fcf0 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8b4402fcf0 msgr2=0x7f8b440321b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8b4402fcf0 0x7f8b440321b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f8b4c0096a0 tx=0x7f8b4c006b40).stop 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 msgr2=0x7f8b6410cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b6410cc00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8b5400b190 tx=0x7f8b54006ee0).stop 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 shutdown_connections 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8b4402fcf0 0x7f8b440321b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 --2- 172.21.15.55:0/1608193406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b64070390 0x7f8b6410cc00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.713+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 >> 172.21.15.55:0/1608193406 conn(0x7f8b640f63c0 msgr2=0x7f8b640f7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.714+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 shutdown_connections 2023-12-06T03:20:48.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:48.714+0000 7f8b6a694700 1 -- 172.21.15.55:0/1608193406 wait complete. 2023-12-06T03:20:49.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:48 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:47.748465+0000 mgr.smithi055.qjkkyb (mgr.14116) 1 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-06T03:20:49.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:48 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:47.749005+0000 mon.smithi055 (mon.0) 49 : cluster [DBG] mgrmap e7: smithi055.qjkkyb(active, since 1.06421s) 2023-12-06T03:20:49.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:48 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:47.755520+0000 mgr.smithi055.qjkkyb (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-06T03:20:49.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:48 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:48.711955+0000 mon.smithi055 (mon.0) 50 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:49.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:48 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:48.716284+0000 mon.smithi055 (mon.0) 51 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:20:50.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:48.705174+0000 mgr.smithi055.qjkkyb (mgr.14116) 3 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:20:50.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:20:48.750339+0000 mon.smithi055 (mon.0) 52 : cluster [DBG] mgrmap e8: smithi055.qjkkyb(active, since 2s) 2023-12-06T03:20:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm set-user root 2023-12-06T03:20:52.448 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c8384700 1 Processor -- start 2023-12-06T03:20:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c8384700 1 -- start start 2023-12-06T03:20:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c8384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c00fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c8384700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c00fb490 con 0x7fa4c00fab80 2023-12-06T03:20:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c6120700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c00fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c6120700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c00fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37946/0 (socket says 172.21.15.55:37946) 2023-12-06T03:20:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.177+0000 7fa4c6120700 1 -- 172.21.15.55:0/2682147836 learned_addr learned my addr 172.21.15.55:0/2682147836 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c6120700 1 -- 172.21.15.55:0/2682147836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c00fb5d0 con 0x7fa4c00fab80 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c6120700 1 --2- 172.21.15.55:0/2682147836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c00fafa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa4bc00b5f0 tx=0x7fa4bc006ee0).ready entity=mon.0 client_cookie=1f3017679fe15db5 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c511e700 1 -- 172.21.15.55:0/2682147836 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4bc00d600 con 0x7fa4c00fab80 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c511e700 1 -- 172.21.15.55:0/2682147836 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fa4bc00dbd0 con 0x7fa4c00fab80 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c511e700 1 -- 172.21.15.55:0/2682147836 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4bc011c00 con 0x7fa4c00fab80 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c8384700 1 -- 172.21.15.55:0/2682147836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 msgr2=0x7fa4c00fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c8384700 1 --2- 172.21.15.55:0/2682147836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c00fafa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa4bc00b5f0 tx=0x7fa4bc006ee0).stop 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c8384700 1 -- 172.21.15.55:0/2682147836 shutdown_connections 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.178+0000 7fa4c8384700 1 --2- 172.21.15.55:0/2682147836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c00fafa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 -- 172.21.15.55:0/2682147836 >> 172.21.15.55:0/2682147836 conn(0x7fa4c00f63c0 msgr2=0x7fa4c00f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 -- 172.21.15.55:0/2682147836 shutdown_connections 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 -- 172.21.15.55:0/2682147836 wait complete. 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 Processor -- start 2023-12-06T03:20:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 -- start start 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c010d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c8384700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c0070450 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c6120700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c010d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c6120700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c010d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37952/0 (socket says 172.21.15.55:37952) 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.179+0000 7fa4c6120700 1 -- 172.21.15.55:0/3282804941 learned_addr learned my addr 172.21.15.55:0/3282804941 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4c6120700 1 -- 172.21.15.55:0/3282804941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc00b200 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4c6120700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c010d720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa4bc00b1d0 tx=0x7fa4bc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4bc01b410 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fa4bc01b9e0 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4bc01a840 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c0070650 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.180+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c010ddd0 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.181+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fa4bc00f070 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.181+0000 7fa4aeffd700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fa4a802fdb0 0x7fa4a8032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.181+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa4bc0411c0 con 0x7fa4c00fab80 2023-12-06T03:20:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.181+0000 7fa4c591f700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fa4a802fdb0 0x7fa4a8032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.181+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c0059100 con 0x7fa4c00fab80 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.182+0000 7fa4c591f700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fa4a802fdb0 0x7fa4a8032270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fa4b00096a0 tx=0x7fa4b0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.186+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa4bc025460 con 0x7fa4c00fab80 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.355+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4c00fc9d0 con 0x7fa4a802fdb0 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.356+0000 7fa4aeffd700 1 -- 172.21.15.55:0/3282804941 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fa4c00fc9d0 con 0x7fa4a802fdb0 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fa4a802fdb0 msgr2=0x7fa4a8032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fa4a802fdb0 0x7fa4a8032270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fa4b00096a0 tx=0x7fa4b0006b40).stop 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 msgr2=0x7fa4c010d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c010d720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa4bc00b1d0 tx=0x7fa4bc006ee0).stop 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 shutdown_connections 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fa4a802fdb0 0x7fa4a8032270 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 --2- 172.21.15.55:0/3282804941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c00fab80 0x7fa4c010d720 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 >> 172.21.15.55:0/3282804941 conn(0x7fa4c00f63c0 msgr2=0x7fa4c00f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 shutdown_connections 2023-12-06T03:20:52.456 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:52.358+0000 7fa4c8384700 1 -- 172.21.15.55:0/3282804941 wait complete. 2023-12-06T03:20:52.457 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: value unchanged 2023-12-06T03:20:53.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:52.357537+0000 mgr.smithi055.qjkkyb (mgr.14116) 4 : audit [DBG] from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:20:54.582 INFO:teuthology.orchestra.run.smithi055.stderr:Generating ssh key... 2023-12-06T03:20:54.583 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm generate-key 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.837+0000 7f3011b32700 1 Processor -- start 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f3011b32700 1 -- start start 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f3011b32700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f3011b32700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0fedd0 con 0x7f300c070430 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f300b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f300b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37954/0 (socket says 172.21.15.55:37954) 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f300b7fe700 1 -- 172.21.15.55:0/884530653 learned_addr learned my addr 172.21.15.55:0/884530653 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f300b7fe700 1 -- 172.21.15.55:0/884530653 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c0fef10 con 0x7f300c070430 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f300b7fe700 1 --2- 172.21.15.55:0/884530653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fe8e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2ffc00b5f0 tx=0x7f2ffc006ee0).ready entity=mon.0 client_cookie=e30474d9e718ce5e server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.838+0000 7f300a7fc700 1 -- 172.21.15.55:0/884530653 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ffc00d600 con 0x7f300c070430 2023-12-06T03:20:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f300a7fc700 1 -- 172.21.15.55:0/884530653 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f2ffc00dbd0 con 0x7f300c070430 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f300a7fc700 1 -- 172.21.15.55:0/884530653 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ffc011c00 con 0x7f300c070430 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 -- 172.21.15.55:0/884530653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 msgr2=0x7f300c0fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 --2- 172.21.15.55:0/884530653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fe8e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f2ffc00b5f0 tx=0x7f2ffc006ee0).stop 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 -- 172.21.15.55:0/884530653 shutdown_connections 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 --2- 172.21.15.55:0/884530653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fe8e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 -- 172.21.15.55:0/884530653 >> 172.21.15.55:0/884530653 conn(0x7f300c0f63c0 msgr2=0x7f300c0f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 -- 172.21.15.55:0/884530653 shutdown_connections 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.839+0000 7f3011b32700 1 -- 172.21.15.55:0/884530653 wait complete. 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f3011b32700 1 Processor -- start 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f3011b32700 1 -- start start 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f3011b32700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f3011b32700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0fa7e0 con 0x7f300c070430 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f300b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f300b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37970/0 (socket says 172.21.15.55:37970) 2023-12-06T03:20:55.953 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.840+0000 7f300b7fe700 1 -- 172.21.15.55:0/3211336135 learned_addr learned my addr 172.21.15.55:0/3211336135 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f300b7fe700 1 -- 172.21.15.55:0/3211336135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc00b200 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f300b7fe700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fc360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2ffc00b190 tx=0x7f2ffc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ffc01b410 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f2ffc01b9e0 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c0fa980 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ffc01a880 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.841+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c0fad90 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.842+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f2ffc00f070 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.842+0000 7f2ff3fff700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f2ff802fd50 0x7f2ff8032210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.842+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f2ffc01bb50 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.842+0000 7f300affd700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f2ff802fd50 0x7f2ff8032210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.842+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c059100 con 0x7f300c070430 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.842+0000 7f300affd700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f2ff802fd50 0x7f2ff8032210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f30000096a0 tx=0x7f3000006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:55.954 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:55.847+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2ffc010360 con 0x7f300c070430 2023-12-06T03:20:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.014+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f300c1048c0 con 0x7f2ff802fd50 2023-12-06T03:20:56.325 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.323+0000 7f2ff3fff700 1 -- 172.21.15.55:0/3211336135 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f300c1048c0 con 0x7f2ff802fd50 2023-12-06T03:20:56.327 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f2ff802fd50 msgr2=0x7f2ff8032210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:56.327 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f2ff802fd50 0x7f2ff8032210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f30000096a0 tx=0x7f3000006b40).stop 2023-12-06T03:20:56.327 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 msgr2=0x7f300c0fc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fc360 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2ffc00b190 tx=0x7f2ffc006ee0).stop 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 shutdown_connections 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f2ff802fd50 0x7f2ff8032210 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 --2- 172.21.15.55:0/3211336135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f300c070430 0x7f300c0fc360 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 >> 172.21.15.55:0/3211336135 conn(0x7f300c0f63c0 msgr2=0x7f300c0f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 shutdown_connections 2023-12-06T03:20:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:56.325+0000 7f3011b32700 1 -- 172.21.15.55:0/3211336135 wait complete. 2023-12-06T03:20:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:20:56.016668+0000 mgr.smithi055.qjkkyb (mgr.14116) 6 : cephadm [INF] Generating ssh key... 2023-12-06T03:20:57.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:56.314303+0000 mon.smithi055 (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:57.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:56.323594+0000 mon.smithi055 (mon.0) 54 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:20:57.782 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm get-pub-key 2023-12-06T03:20:58.142 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f484bff7700 1 Processor -- start 2023-12-06T03:20:58.142 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f484bff7700 1 -- start start 2023-12-06T03:20:58.142 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f484bff7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f48440fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:58.142 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f484bff7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48440fb4a0 con 0x7f48440fab90 2023-12-06T03:20:58.142 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f4849d93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f48440fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f4849d93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f48440fafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37972/0 (socket says 172.21.15.55:37972) 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.140+0000 7f4849d93700 1 -- 172.21.15.55:0/3994456211 learned_addr learned my addr 172.21.15.55:0/3994456211 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.141+0000 7f4849d93700 1 -- 172.21.15.55:0/3994456211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440fb5e0 con 0x7f48440fab90 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.141+0000 7f4849d93700 1 --2- 172.21.15.55:0/3994456211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f48440fafb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f484000b870 tx=0x7f4840006f70).ready entity=mon.0 client_cookie=5ee5494542c72687 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.141+0000 7f4848d91700 1 -- 172.21.15.55:0/3994456211 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f484000da20 con 0x7f48440fab90 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.141+0000 7f4848d91700 1 -- 172.21.15.55:0/3994456211 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f4840010470 con 0x7f48440fab90 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.141+0000 7f4848d91700 1 -- 172.21.15.55:0/3994456211 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f484000f5b0 con 0x7f48440fab90 2023-12-06T03:20:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 -- 172.21.15.55:0/3994456211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 msgr2=0x7f48440fafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 --2- 172.21.15.55:0/3994456211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f48440fafb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f484000b870 tx=0x7f4840006f70).stop 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 -- 172.21.15.55:0/3994456211 shutdown_connections 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 --2- 172.21.15.55:0/3994456211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f48440fafb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 -- 172.21.15.55:0/3994456211 >> 172.21.15.55:0/3994456211 conn(0x7f48440f63b0 msgr2=0x7f48440f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 -- 172.21.15.55:0/3994456211 shutdown_connections 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.142+0000 7f484bff7700 1 -- 172.21.15.55:0/3994456211 wait complete. 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f484bff7700 1 Processor -- start 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f484bff7700 1 -- start start 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f484bff7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f4844108730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f484bff7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844108c20 con 0x7f48440fab90 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f4849d93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f4844108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f4849d93700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f4844108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37988/0 (socket says 172.21.15.55:37988) 2023-12-06T03:20:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f4849d93700 1 -- 172.21.15.55:0/610283382 learned_addr learned my addr 172.21.15.55:0/610283382 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.143+0000 7f4849d93700 1 -- 172.21.15.55:0/610283382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484000b480 con 0x7f48440fab90 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.144+0000 7f4849d93700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f4844108730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4840006a40 tx=0x7f4840006f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.144+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f484000f3d0 con 0x7f48440fab90 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.144+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f484000fef0 con 0x7f48440fab90 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.144+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4844108e20 con 0x7f48440fab90 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.144+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4840019d40 con 0x7f48440fab90 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.144+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4844109230 con 0x7f48440fab90 2023-12-06T03:20:58.148 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.145+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f484002b060 con 0x7f48440fab90 2023-12-06T03:20:58.149 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.145+0000 7f48327fc700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f482c02fdb0 0x7f482c032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:58.149 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.145+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4844045ad0 con 0x7f48440fab90 2023-12-06T03:20:58.149 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.145+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f4840044010 con 0x7f48440fab90 2023-12-06T03:20:58.149 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.145+0000 7f4849592700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f482c02fdb0 0x7f482c032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:58.149 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.146+0000 7f4849592700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f482c02fdb0 0x7f482c032270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f48340096a0 tx=0x7f4834006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:58.151 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.150+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4840014030 con 0x7f48440fab90 2023-12-06T03:20:58.319 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.318+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4844109b10 con 0x7f482c02fdb0 2023-12-06T03:20:58.320 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.318+0000 7f48327fc700 1 -- 172.21.15.55:0/610283382 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4844109b10 con 0x7f482c02fdb0 2023-12-06T03:20:58.320 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC59hmyDdUdG6zA//0cDdjwFDZenhT3qyJ0LflT/m9Ww296uBah/pXwb0zmfwMJBZ/aFdTWI839qUokYof+LYFdOfVvR6aBc1AzYDFWj0UgOEM14xMDieAvNIbVhRqFaJz/DJO0LASJV1HscS0gWUBIka0zAgqLNueNZH0l7rcf6eVGprjz8iFTfn2OaXswQu92Xhi87HYFKyzRYIZr/SsbJSOBNggo65iq52pPgbJx0iex7qobYCyN0MngFYgPTMBZr/6ZnjH2+CYnPDsn3DbFTcE6eUAX/tWBcEm+FQrZZm1Ohbg9FOHD8iizhKiGji3UGo3pCHMw2OgBEKAzxSg3IspJ3iVnfj+yUqt0QotsdQ/GAh8YntjgGQoqjUzcCcrNjM8CpzOOkt/N3VMYkxe9YQgUw2cDR5G+xpeNJf8fjF5oiTVYj3wtqSJtf4BLWznjharzUjaXGIXdTzYQ6HB3PjC15XmF/KVBG8gveuN2U7r5nZKc1RRSZnw2foY7/30= ceph-5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:20:58.322 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f482c02fdb0 msgr2=0x7f482c032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f482c02fdb0 0x7f482c032270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f48340096a0 tx=0x7f4834006b40).stop 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 msgr2=0x7f4844108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f4844108730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f4840006a40 tx=0x7f4840006f70).stop 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 shutdown_connections 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f482c02fdb0 0x7f482c032270 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 --2- 172.21.15.55:0/610283382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48440fab90 0x7f4844108730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 >> 172.21.15.55:0/610283382 conn(0x7f48440f63b0 msgr2=0x7f48440ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 shutdown_connections 2023-12-06T03:20:58.323 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:58.320+0000 7f484bff7700 1 -- 172.21.15.55:0/610283382 wait complete. 2023-12-06T03:20:58.674 INFO:teuthology.orchestra.run.smithi055.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-06T03:20:58.674 INFO:teuthology.orchestra.run.smithi055.stderr:Adding key to root@localhost authorized_keys... 2023-12-06T03:20:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:Adding host smithi055... 2023-12-06T03:20:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch host add smithi055 172.21.15.55 2023-12-06T03:20:59.065 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.064+0000 7f7fa3c7b700 1 Processor -- start 2023-12-06T03:20:59.066 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.065+0000 7f7fa3c7b700 1 -- start start 2023-12-06T03:20:59.066 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.065+0000 7f7fa3c7b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c0fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:59.066 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.065+0000 7f7fa3c7b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0fc5b0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.065+0000 7f7fa1a17700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c0fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.065+0000 7f7fa1a17700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c0fc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37996/0 (socket says 172.21.15.55:37996) 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.065+0000 7f7fa1a17700 1 -- 172.21.15.55:0/2023275359 learned_addr learned my addr 172.21.15.55:0/2023275359 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.066+0000 7f7fa1a17700 1 -- 172.21.15.55:0/2023275359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0fc6f0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.066+0000 7f7fa1a17700 1 --2- 172.21.15.55:0/2023275359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c0fc0c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7f9800b5f0 tx=0x7f7f98006ee0).ready entity=mon.0 client_cookie=a7d86ded44d2c6dd server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.066+0000 7f7fa0a15700 1 -- 172.21.15.55:0/2023275359 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9800d600 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.066+0000 7f7fa0a15700 1 -- 172.21.15.55:0/2023275359 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f7f9800dbd0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.066+0000 7f7fa0a15700 1 -- 172.21.15.55:0/2023275359 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f98011c00 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2023275359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 msgr2=0x7f7f9c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:59.068 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 --2- 172.21.15.55:0/2023275359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c0fc0c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7f9800b5f0 tx=0x7f7f98006ee0).stop 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2023275359 shutdown_connections 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 --2- 172.21.15.55:0/2023275359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c0fc0c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2023275359 >> 172.21.15.55:0/2023275359 conn(0x7f7f9c0f74c0 msgr2=0x7f7f9c0f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2023275359 shutdown_connections 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2023275359 wait complete. 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 Processor -- start 2023-12-06T03:20:59.069 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.067+0000 7f7fa3c7b700 1 -- start start 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa3c7b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa3c7b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0709f0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa1a17700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa1a17700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38010/0 (socket says 172.21.15.55:38010) 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa1a17700 1 -- 172.21.15.55:0/2608780276 learned_addr learned my addr 172.21.15.55:0/2608780276 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa1a17700 1 -- 172.21.15.55:0/2608780276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9800b200 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7fa1a17700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c072570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7f9800b1d0 tx=0x7f7f98006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.068+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9801b410 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.069+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f7f9801b9e0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.069+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c070b90 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.071 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.069+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9801a840 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.072 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.069+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c070fa0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.072 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.070+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c045ad0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.072 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.070+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f7f9800f070 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.072 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.070+0000 7f7f8e7fc700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f7f8402fdb0 0x7f7f84032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:20:59.073 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.070+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f7f980411a0 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.073 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.070+0000 7f7fa1216700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f7f8402fdb0 0x7f7f84032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:20:59.073 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.070+0000 7f7fa1216700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f7f8402fdb0 0x7f7f84032270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f7f900096a0 tx=0x7f7f90006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:20:59.075 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.075+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7f98019950 con 0x7f7f9c0fbca0 2023-12-06T03:20:59.265 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.264+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi055", "addr": "172.21.15.55", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f9c0fdb30 con 0x7f7f8402fdb0 2023-12-06T03:20:59.920 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Added host 'smithi055' 2023-12-06T03:20:59.921 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.919+0000 7f7f8e7fc700 1 -- 172.21.15.55:0/2608780276 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7f7f9c0fdb30 con 0x7f7f8402fdb0 2023-12-06T03:20:59.923 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.921+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f7f8402fdb0 msgr2=0x7f7f84032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:59.923 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.921+0000 7f7fa3c7b700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f7f8402fdb0 0x7f7f84032270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f7f900096a0 tx=0x7f7f90006b40).stop 2023-12-06T03:20:59.923 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 msgr2=0x7f7f9c072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:20:59.923 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c072570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7f9800b1d0 tx=0x7f7f98006ee0).stop 2023-12-06T03:20:59.923 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 shutdown_connections 2023-12-06T03:20:59.923 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f7f8402fdb0 0x7f7f84032270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:59.924 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 --2- 172.21.15.55:0/2608780276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f9c0fbca0 0x7f7f9c072570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:20:59.924 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 >> 172.21.15.55:0/2608780276 conn(0x7f7f9c0f74c0 msgr2=0x7f7f9c0f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:20:59.924 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 shutdown_connections 2023-12-06T03:20:59.924 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:20:59.922+0000 7f7fa3c7b700 1 -- 172.21.15.55:0/2608780276 wait complete. 2023-12-06T03:21:00.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:58.319714+0000 mgr.smithi055.qjkkyb (mgr.14116) 7 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:00.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:59.918792+0000 mon.smithi055 (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:00.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:20:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:59.919433+0000 mon.smithi055 (mon.0) 56 : audit [DBG] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:21:00.216 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying mon service with default placement... 2023-12-06T03:21:00.216 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mon 2023-12-06T03:21:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.598+0000 7f8cd00eb700 1 Processor -- start 2023-12-06T03:21:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.599+0000 7f8cd00eb700 1 -- start start 2023-12-06T03:21:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.599+0000 7f8cd00eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.599+0000 7f8cd00eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80fedd0 con 0x7f8cc8070430 2023-12-06T03:21:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.599+0000 7f8ccde87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccde87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33708/0 (socket says 172.21.15.55:33708) 2023-12-06T03:21:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccde87700 1 -- 172.21.15.55:0/2297280058 learned_addr learned my addr 172.21.15.55:0/2297280058 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccde87700 1 -- 172.21.15.55:0/2297280058 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc80fef10 con 0x7f8cc8070430 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccde87700 1 --2- 172.21.15.55:0/2297280058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fe8e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f8cc400b5f0 tx=0x7f8cc4006ee0).ready entity=mon.0 client_cookie=209dc4bf294940f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccce85700 1 -- 172.21.15.55:0/2297280058 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc400d600 con 0x7f8cc8070430 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccce85700 1 -- 172.21.15.55:0/2297280058 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8cc400dbd0 con 0x7f8cc8070430 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.600+0000 7f8ccce85700 1 -- 172.21.15.55:0/2297280058 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc4011c00 con 0x7f8cc8070430 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 -- 172.21.15.55:0/2297280058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 msgr2=0x7f8cc80fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 --2- 172.21.15.55:0/2297280058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fe8e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f8cc400b5f0 tx=0x7f8cc4006ee0).stop 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 -- 172.21.15.55:0/2297280058 shutdown_connections 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 --2- 172.21.15.55:0/2297280058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fe8e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 -- 172.21.15.55:0/2297280058 >> 172.21.15.55:0/2297280058 conn(0x7f8cc80f63c0 msgr2=0x7f8cc80f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 -- 172.21.15.55:0/2297280058 shutdown_connections 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 -- 172.21.15.55:0/2297280058 wait complete. 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.601+0000 7f8cd00eb700 1 Processor -- start 2023-12-06T03:21:00.603 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8cd00eb700 1 -- start start 2023-12-06T03:21:00.604 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8cd00eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:00.604 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8cd00eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80fa7e0 con 0x7f8cc8070430 2023-12-06T03:21:00.604 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8ccde87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:00.604 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8ccde87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33716/0 (socket says 172.21.15.55:33716) 2023-12-06T03:21:00.604 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8ccde87700 1 -- 172.21.15.55:0/183945035 learned_addr learned my addr 172.21.15.55:0/183945035 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:00.605 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8ccde87700 1 -- 172.21.15.55:0/183945035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc400b200 con 0x7f8cc8070430 2023-12-06T03:21:00.605 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8ccde87700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fc360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8cc400b190 tx=0x7f8cc4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:00.605 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.602+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc401b410 con 0x7f8cc8070430 2023-12-06T03:21:00.605 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.603+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8cc401b9e0 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.603+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc80fa980 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.603+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cc401a880 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.603+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc80fad90 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.604+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f8cc400f070 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.604+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc8045ad0 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.604+0000 7f8cb67fc700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8cb002fdb0 0x7f8cb0032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.604+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f8cc40411d0 con 0x7f8cc8070430 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.604+0000 7f8ccd686700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8cb002fdb0 0x7f8cb0032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:00.606 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.604+0000 7f8ccd686700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8cb002fdb0 0x7f8cb0032270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f8cb80096a0 tx=0x7f8cb8006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.608+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8cc40224f0 con 0x7f8cc8070430 2023-12-06T03:21:00.802 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.801+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cc80fafd0 con 0x7f8cb002fdb0 2023-12-06T03:21:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.810+0000 7f8cb67fc700 1 -- 172.21.15.55:0/183945035 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8cc80fafd0 con 0x7f8cb002fdb0 2023-12-06T03:21:00.811 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled mon update... 2023-12-06T03:21:00.813 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8cb002fdb0 msgr2=0x7f8cb0032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8cb002fdb0 0x7f8cb0032270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f8cb80096a0 tx=0x7f8cb8006b40).stop 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 msgr2=0x7f8cc80fc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fc360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f8cc400b190 tx=0x7f8cc4006ee0).stop 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 shutdown_connections 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f8cb002fdb0 0x7f8cb0032270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 --2- 172.21.15.55:0/183945035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cc8070430 0x7f8cc80fc360 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 >> 172.21.15.55:0/183945035 conn(0x7f8cc80f63c0 msgr2=0x7f8cc80f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 shutdown_connections 2023-12-06T03:21:00.814 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:00.812+0000 7f8cd00eb700 1 -- 172.21.15.55:0/183945035 wait complete. 2023-12-06T03:21:01.157 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying mgr service with default placement... 2023-12-06T03:21:01.157 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mgr 2023-12-06T03:21:01.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:00 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:20:59.265573+0000 mgr.smithi055.qjkkyb (mgr.14116) 8 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi055", "addr": "172.21.15.55", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:01.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:00 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:20:59.919149+0000 mgr.smithi055.qjkkyb (mgr.14116) 9 : cephadm [INF] Added host smithi055 2023-12-06T03:21:01.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:00 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:00.810415+0000 mon.smithi055 (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:01.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.563+0000 7f3a1d7f7700 1 Processor -- start 2023-12-06T03:21:01.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.563+0000 7f3a1d7f7700 1 -- start start 2023-12-06T03:21:01.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a1d7f7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 0x7f3a180fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:01.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a1d7f7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a180fc5c0 con 0x7f3a180fbcb0 2023-12-06T03:21:01.565 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 0x7f3a180fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:01.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 0x7f3a180fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33722/0 (socket says 172.21.15.55:33722) 2023-12-06T03:21:01.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a16ffd700 1 -- 172.21.15.55:0/2385355094 learned_addr learned my addr 172.21.15.55:0/2385355094 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:01.566 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a16ffd700 1 -- 172.21.15.55:0/2385355094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180fc700 con 0x7f3a180fbcb0 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.564+0000 7f3a16ffd700 1 --2- 172.21.15.55:0/2385355094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 0x7f3a180fc0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3a0800b5f0 tx=0x7f3a08006ee0).ready entity=mon.0 client_cookie=e7bda4d90831369 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.565+0000 7f3a15ffb700 1 -- 172.21.15.55:0/2385355094 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a0800d600 con 0x7f3a180fbcb0 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.565+0000 7f3a15ffb700 1 -- 172.21.15.55:0/2385355094 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f3a0800dbd0 con 0x7f3a180fbcb0 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.565+0000 7f3a15ffb700 1 -- 172.21.15.55:0/2385355094 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a08011c00 con 0x7f3a180fbcb0 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.565+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/2385355094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 msgr2=0x7f3a180fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.565+0000 7f3a1d7f7700 1 --2- 172.21.15.55:0/2385355094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 0x7f3a180fc0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3a0800b5f0 tx=0x7f3a08006ee0).stop 2023-12-06T03:21:01.567 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.566+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/2385355094 shutdown_connections 2023-12-06T03:21:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.566+0000 7f3a1d7f7700 1 --2- 172.21.15.55:0/2385355094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a180fbcb0 0x7f3a180fc0d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.566+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/2385355094 >> 172.21.15.55:0/2385355094 conn(0x7f3a180f74b0 msgr2=0x7f3a180f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.566+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/2385355094 shutdown_connections 2023-12-06T03:21:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.566+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/2385355094 wait complete. 2023-12-06T03:21:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a1d7f7700 1 Processor -- start 2023-12-06T03:21:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a1d7f7700 1 -- start start 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a1d7f7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 0x7f3a18070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a1d7f7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a18072950 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 0x7f3a18070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 0x7f3a18070a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33730/0 (socket says 172.21.15.55:33730) 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.567+0000 7f3a16ffd700 1 -- 172.21.15.55:0/37267309 learned_addr learned my addr 172.21.15.55:0/37267309 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f3a16ffd700 1 -- 172.21.15.55:0/37267309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0800b200 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f3a16ffd700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 0x7f3a18070a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3a08011820 tx=0x7f3a08006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a0800d9f0 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f3a0801a9f0 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a18070f50 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a0801b790 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.568+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a18071360 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.569+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f3a0800f070 con 0x7f3a18072530 2023-12-06T03:21:01.570 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.569+0000 7f39ff7fe700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f3a0402fdb0 0x7f3a04032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:01.571 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.569+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f3a080412d0 con 0x7f3a18072530 2023-12-06T03:21:01.571 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.569+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a18045ad0 con 0x7f3a18072530 2023-12-06T03:21:01.571 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.569+0000 7f3a167fc700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f3a0402fdb0 0x7f3a04032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:01.571 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.570+0000 7f3a167fc700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f3a0402fdb0 0x7f3a04032270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f3a0c0096a0 tx=0x7f3a0c006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:01.574 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.574+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3a08016070 con 0x7f3a18072530 2023-12-06T03:21:01.767 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.767+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a18057f10 con 0x7f3a0402fdb0 2023-12-06T03:21:01.776 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.776+0000 7f39ff7fe700 1 -- 172.21.15.55:0/37267309 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3a18057f10 con 0x7f3a0402fdb0 2023-12-06T03:21:01.776 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled mgr update... 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f3a0402fdb0 msgr2=0x7f3a04032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f3a0402fdb0 0x7f3a04032270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f3a0c0096a0 tx=0x7f3a0c006b40).stop 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 msgr2=0x7f3a18070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 0x7f3a18070a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3a08011820 tx=0x7f3a08006ee0).stop 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 shutdown_connections 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7f3a0402fdb0 0x7f3a04032270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 --2- 172.21.15.55:0/37267309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a18072530 0x7f3a18070a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 >> 172.21.15.55:0/37267309 conn(0x7f3a180f74b0 msgr2=0x7f3a180f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 shutdown_connections 2023-12-06T03:21:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:01.778+0000 7f3a1d7f7700 1 -- 172.21.15.55:0/37267309 wait complete. 2023-12-06T03:21:02.093 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:00.802652+0000 mgr.smithi055.qjkkyb (mgr.14116) 10 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:02.093 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:00.803485+0000 mgr.smithi055.qjkkyb (mgr.14116) 11 : cephadm [INF] Saving service mon spec with placement count:5 2023-12-06T03:21:02.093 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:01.776109+0000 mon.smithi055 (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:02.140 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying crash service with default placement... 2023-12-06T03:21:02.140 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply crash 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.612+0000 7fabbb59e700 1 Processor -- start 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.612+0000 7fabbb59e700 1 -- start start 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.612+0000 7fabbb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.612+0000 7fabbb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0fec80 con 0x7fabbc0703d0 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33740/0 (socket says 172.21.15.55:33740) 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabba59c700 1 -- 172.21.15.55:0/4290972527 learned_addr learned my addr 172.21.15.55:0/4290972527 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:02.614 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabba59c700 1 -- 172.21.15.55:0/4290972527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0fedc0 con 0x7fabbc0703d0 2023-12-06T03:21:02.616 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabba59c700 1 --2- 172.21.15.55:0/4290972527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fe790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fabac00b5f0 tx=0x7fabac006ee0).ready entity=mon.0 client_cookie=ad73ce9066306661 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:02.616 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabb959a700 1 -- 172.21.15.55:0/4290972527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabac00d600 con 0x7fabbc0703d0 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabb959a700 1 -- 172.21.15.55:0/4290972527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fabac00dbd0 con 0x7fabbc0703d0 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.613+0000 7fabb959a700 1 -- 172.21.15.55:0/4290972527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabac011c00 con 0x7fabbc0703d0 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 -- 172.21.15.55:0/4290972527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 msgr2=0x7fabbc0fe790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 --2- 172.21.15.55:0/4290972527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fe790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fabac00b5f0 tx=0x7fabac006ee0).stop 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 -- 172.21.15.55:0/4290972527 shutdown_connections 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 --2- 172.21.15.55:0/4290972527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fe790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 -- 172.21.15.55:0/4290972527 >> 172.21.15.55:0/4290972527 conn(0x7fabbc0f6210 msgr2=0x7fabbc0f8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 -- 172.21.15.55:0/4290972527 shutdown_connections 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.614+0000 7fabbb59e700 1 -- 172.21.15.55:0/4290972527 wait complete. 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.615+0000 7fabbb59e700 1 Processor -- start 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.615+0000 7fabbb59e700 1 -- start start 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.615+0000 7fabbb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.615+0000 7fabbb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0fa690 con 0x7fabbc0703d0 2023-12-06T03:21:02.617 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabba59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33756/0 (socket says 172.21.15.55:33756) 2023-12-06T03:21:02.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabba59c700 1 -- 172.21.15.55:0/3134877477 learned_addr learned my addr 172.21.15.55:0/3134877477 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabba59c700 1 -- 172.21.15.55:0/3134877477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac00b200 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabba59c700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fc210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fabac00b190 tx=0x7fabac006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabac01b410 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fabac01b9e0 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fabac01a880 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0fa830 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.616+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc0fac40 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.617+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fabac00f070 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.617+0000 7fabaaffd700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7faba002fdb0 0x7faba0032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.617+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc045ad0 con 0x7fabbc0703d0 2023-12-06T03:21:02.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.617+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fabac0412d0 con 0x7fabbc0703d0 2023-12-06T03:21:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.618+0000 7fabb9d9b700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7faba002fdb0 0x7faba0032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:02.620 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.618+0000 7fabb9d9b700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7faba002fdb0 0x7faba0032270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fabb00096a0 tx=0x7fabb0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:02.622 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.622+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fabac016070 con 0x7fabbc0703d0 2023-12-06T03:21:02.823 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.822+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fabbc0fae80 con 0x7faba002fdb0 2023-12-06T03:21:02.827 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.827+0000 7fabaaffd700 1 -- 172.21.15.55:0/3134877477 <== mgr.14116 v2:172.21.15.55:6800/3025569119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fabbc0fae80 con 0x7faba002fdb0 2023-12-06T03:21:02.827 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled crash update... 2023-12-06T03:21:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7faba002fdb0 msgr2=0x7faba0032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7faba002fdb0 0x7faba0032270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fabb00096a0 tx=0x7fabb0006b40).stop 2023-12-06T03:21:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 msgr2=0x7fabbc0fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:02.830 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fc210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fabac00b190 tx=0x7fabac006ee0).stop 2023-12-06T03:21:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 shutdown_connections 2023-12-06T03:21:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7faba002fdb0 0x7faba0032270 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 --2- 172.21.15.55:0/3134877477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabbc0703d0 0x7fabbc0fc210 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 >> 172.21.15.55:0/3134877477 conn(0x7fabbc0f6210 msgr2=0x7fabbc0f7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 shutdown_connections 2023-12-06T03:21:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:02.829+0000 7fabbb59e700 1 -- 172.21.15.55:0/3134877477 wait complete. 2023-12-06T03:21:03.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:02 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:01.768017+0000 mgr.smithi055.qjkkyb (mgr.14116) 12 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:03.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:02 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:01.768564+0000 mgr.smithi055.qjkkyb (mgr.14116) 13 : cephadm [INF] Saving service mgr spec with placement count:2 2023-12-06T03:21:03.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:02 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:02.826886+0000 mon.smithi055 (mon.0) 59 : audit [INF] from='mgr.14116 172.21.15.55:0/2105001048' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:03.248 INFO:teuthology.orchestra.run.smithi055.stderr:Enabling mgr prometheus module... 2023-12-06T03:21:03.248 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable prometheus 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf075ad700 1 Processor -- start 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf075ad700 1 -- start start 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf075ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf075ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000fdc30 con 0x7fbf000fd320 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf05349700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf05349700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33764/0 (socket says 172.21.15.55:33764) 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.646+0000 7fbf05349700 1 -- 172.21.15.55:0/1304028753 learned_addr learned my addr 172.21.15.55:0/1304028753 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.647+0000 7fbf05349700 1 -- 172.21.15.55:0/1304028753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000fdd70 con 0x7fbf000fd320 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.647+0000 7fbf05349700 1 --2- 172.21.15.55:0/1304028753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fd740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fbefc00b5f0 tx=0x7fbefc006ee0).ready entity=mon.0 client_cookie=3e01772c2c68fb44 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.647+0000 7fbeeffff700 1 -- 172.21.15.55:0/1304028753 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc00d600 con 0x7fbf000fd320 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.647+0000 7fbeeffff700 1 -- 172.21.15.55:0/1304028753 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fbefc00dbd0 con 0x7fbf000fd320 2023-12-06T03:21:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.647+0000 7fbeeffff700 1 -- 172.21.15.55:0/1304028753 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc011c00 con 0x7fbf000fd320 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 -- 172.21.15.55:0/1304028753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 msgr2=0x7fbf000fd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 --2- 172.21.15.55:0/1304028753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fd740 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fbefc00b5f0 tx=0x7fbefc006ee0).stop 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 -- 172.21.15.55:0/1304028753 shutdown_connections 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 --2- 172.21.15.55:0/1304028753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fd740 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 -- 172.21.15.55:0/1304028753 >> 172.21.15.55:0/1304028753 conn(0x7fbf000f5b40 msgr2=0x7fbf000f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 -- 172.21.15.55:0/1304028753 shutdown_connections 2023-12-06T03:21:03.650 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 -- 172.21.15.55:0/1304028753 wait complete. 2023-12-06T03:21:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.648+0000 7fbf075ad700 1 Processor -- start 2023-12-06T03:21:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf075ad700 1 -- start start 2023-12-06T03:21:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf075ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf075ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000fb070 con 0x7fbf000fd320 2023-12-06T03:21:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf05349700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf05349700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33776/0 (socket says 172.21.15.55:33776) 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf05349700 1 -- 172.21.15.55:0/2757639140 learned_addr learned my addr 172.21.15.55:0/2757639140 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf05349700 1 -- 172.21.15.55:0/2757639140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc00b200 con 0x7fbf000fd320 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbf05349700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fcbf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fbefc002f40 tx=0x7fbefc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.649+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc01b410 con 0x7fbf000fd320 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.650+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fbefc01b9e0 con 0x7fbf000fd320 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.650+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf000fb210 con 0x7fbf000fd320 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.650+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc01a8b0 con 0x7fbf000fd320 2023-12-06T03:21:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.650+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf000fb620 con 0x7fbf000fd320 2023-12-06T03:21:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.651+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fbefc00f070 con 0x7fbf000fd320 2023-12-06T03:21:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.651+0000 7fbeedffb700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fbee802fdb0 0x7fbee8032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.651+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fbefc041100 con 0x7fbf000fd320 2023-12-06T03:21:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.651+0000 7fbf04b48700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fbee802fdb0 0x7fbee8032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.651+0000 7fbf04b48700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fbee802fdb0 0x7fbee8032270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fbef00096a0 tx=0x7fbef0006b40).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.651+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00059100 con 0x7fbf000fd320 2023-12-06T03:21:03.656 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.656+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbefc016070 con 0x7fbf000fd320 2023-12-06T03:21:03.863 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.863+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "prometheus"} v 0) v1 -- 0x7fbf00045ad0 con 0x7fbf000fd320 2023-12-06T03:21:03.978 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.977+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7fbefc0257e0 con 0x7fbf000fd320 2023-12-06T03:21:03.979 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.978+0000 7fbeedffb700 1 -- 172.21.15.55:0/2757639140 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "prometheus"}]=0 v9) v1 ==== 89+0+0 (secure 0 0 0) 0x7fbefc0107f0 con 0x7fbf000fd320 2023-12-06T03:21:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.980+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fbee802fdb0 msgr2=0x7fbee8032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.980+0000 7fbf075ad700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fbee802fdb0 0x7fbee8032270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fbef00096a0 tx=0x7fbef0006b40).stop 2023-12-06T03:21:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.980+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 msgr2=0x7fbf000fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:03.981 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.980+0000 7fbf075ad700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fcbf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fbefc002f40 tx=0x7fbefc006ee0).stop 2023-12-06T03:21:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.981+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 shutdown_connections 2023-12-06T03:21:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.981+0000 7fbf075ad700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fbee802fdb0 0x7fbee8032270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.981+0000 7fbf075ad700 1 --2- 172.21.15.55:0/2757639140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf000fd320 0x7fbf000fcbf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:03.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.981+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 >> 172.21.15.55:0/2757639140 conn(0x7fbf000f5b40 msgr2=0x7fbf000ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:03.983 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.981+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 shutdown_connections 2023-12-06T03:21:03.983 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:03.981+0000 7fbf075ad700 1 -- 172.21.15.55:0/2757639140 wait complete. 2023-12-06T03:21:04.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:02.823714+0000 mgr.smithi055.qjkkyb (mgr.14116) 14 : audit [DBG] from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:04.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:02.824415+0000 mgr.smithi055.qjkkyb (mgr.14116) 15 : cephadm [INF] Saving service crash spec with placement * 2023-12-06T03:21:04.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:03.863591+0000 mon.smithi055 (mon.0) 60 : audit [INF] from='client.? 172.21.15.55:0/2757639140' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2023-12-06T03:21:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying prometheus service with default placement... 2023-12-06T03:21:04.340 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply prometheus 2023-12-06T03:21:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc64688a700 1 Processor -- start 2023-12-06T03:21:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc64688a700 1 -- start start 2023-12-06T03:21:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc64688a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc640070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:04.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc64688a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6400728d0 con 0x7fc6400724b0 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc640070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc640070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33808/0 (socket says 172.21.15.55:33808) 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.678+0000 7fc63ffff700 1 -- 172.21.15.55:0/82989686 learned_addr learned my addr 172.21.15.55:0/82989686 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.679+0000 7fc63ffff700 1 -- 172.21.15.55:0/82989686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640070f40 con 0x7fc6400724b0 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.679+0000 7fc63ffff700 1 --2- 172.21.15.55:0/82989686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc640070a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fc63000b5f0 tx=0x7fc630006ee0).ready entity=mon.0 client_cookie=8eeac31d71b6dd2b server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.679+0000 7fc63effd700 1 -- 172.21.15.55:0/82989686 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63000d600 con 0x7fc6400724b0 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.679+0000 7fc63effd700 1 -- 172.21.15.55:0/82989686 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fc63000dbd0 con 0x7fc6400724b0 2023-12-06T03:21:04.681 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.679+0000 7fc63effd700 1 -- 172.21.15.55:0/82989686 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc630011c00 con 0x7fc6400724b0 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 -- 172.21.15.55:0/82989686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 msgr2=0x7fc640070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 --2- 172.21.15.55:0/82989686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc640070a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fc63000b5f0 tx=0x7fc630006ee0).stop 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 -- 172.21.15.55:0/82989686 shutdown_connections 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 --2- 172.21.15.55:0/82989686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc640070a50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 -- 172.21.15.55:0/82989686 >> 172.21.15.55:0/82989686 conn(0x7fc6400f74c0 msgr2=0x7fc6400f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 -- 172.21.15.55:0/82989686 shutdown_connections 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.680+0000 7fc64688a700 1 -- 172.21.15.55:0/82989686 wait complete. 2023-12-06T03:21:04.682 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc64688a700 1 Processor -- start 2023-12-06T03:21:04.683 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc64688a700 1 -- start start 2023-12-06T03:21:04.683 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc64688a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc64010cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc64688a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64010d0e0 con 0x7fc6400724b0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc64010cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc63ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc64010cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33812/0 (socket says 172.21.15.55:33812) 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.681+0000 7fc63ffff700 1 -- 172.21.15.55:0/171985931 learned_addr learned my addr 172.21.15.55:0/171985931 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc63ffff700 1 -- 172.21.15.55:0/171985931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63000b200 con 0x7fc6400724b0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc63ffff700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc64010cbf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc63000b190 tx=0x7fc630006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63001b410 con 0x7fc6400724b0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64010d2e0 con 0x7fc6400724b0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fc63001b9e0 con 0x7fc6400724b0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64010d6f0 con 0x7fc6400724b0 2023-12-06T03:21:04.684 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.682+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63001a880 con 0x7fc6400724b0 2023-12-06T03:21:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.683+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7fc63000f070 con 0x7fc6400724b0 2023-12-06T03:21:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.684+0000 7fc63cff9700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.684+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fc630041200 con 0x7fc6400724b0 2023-12-06T03:21:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.684+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc640045ad0 con 0x7fc6400724b0 2023-12-06T03:21:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.684+0000 7fc63f7fe700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 msgr2=0x7fc62c0322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3025569119 2023-12-06T03:21:04.685 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.684+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-12-06T03:21:04.689 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.688+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc630016070 con 0x7fc6400724b0 2023-12-06T03:21:04.884 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.884+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 --> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fc640106a70 con 0x7fc62c02fe00 2023-12-06T03:21:04.885 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.885+0000 7fc63f7fe700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 msgr2=0x7fc62c0322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3025569119 2023-12-06T03:21:04.885 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:04.885+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-12-06T03:21:05.286 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:05.285+0000 7fc63f7fe700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 msgr2=0x7fc62c0322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3025569119 2023-12-06T03:21:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:05.285+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-12-06T03:21:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:03.978687+0000 mon.smithi055 (mon.0) 61 : audit [INF] from='client.? 172.21.15.55:0/2757639140' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2023-12-06T03:21:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:03.978749+0000 mon.smithi055 (mon.0) 62 : cluster [DBG] mgrmap e9: smithi055.qjkkyb(active, since 17s) 2023-12-06T03:21:06.087 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:06.086+0000 7fc63f7fe700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 msgr2=0x7fc62c0322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3025569119 2023-12-06T03:21:06.088 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:06.086+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-12-06T03:21:07.689 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:07.687+0000 7fc63f7fe700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 msgr2=0x7fc62c0322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3025569119 2023-12-06T03:21:07.689 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:07.687+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-12-06T03:21:09.968 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:09.965+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 10) v1 ==== 35984+0+0 (secure 0 0 0) 0x7fc6300257e0 con 0x7fc6400724b0 2023-12-06T03:21:09.969 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:09.965+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 msgr2=0x7fc62c0322c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-06T03:21:09.969 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:09.965+0000 7fc63cff9700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:21:10.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:09.650207+0000 mon.smithi055 (mon.0) 63 : cluster [INF] Active manager daemon smithi055.qjkkyb restarted 2023-12-06T03:21:10.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:09.650678+0000 mon.smithi055 (mon.0) 64 : cluster [INF] Activating manager daemon smithi055.qjkkyb 2023-12-06T03:21:10.223 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:09.661839+0000 mon.smithi055 (mon.0) 65 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2023-12-06T03:21:10.970 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.967+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 11) v1 ==== 36054+0+0 (secure 0 0 0) 0x7fc6300224f0 con 0x7fc6400724b0 2023-12-06T03:21:10.970 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.967+0000 7fc63cff9700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc62c033c20 0x7fc62c036010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:10.970 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.967+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 --> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fc640106a70 con 0x7fc62c033c20 2023-12-06T03:21:10.971 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.968+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc62c033c20 0x7fc62c036010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:10.971 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.968+0000 7fc63f7fe700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc62c033c20 0x7fc62c036010 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fc634006fd0 tx=0x7fc634006d30).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:10.979 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.977+0000 7fc63cff9700 1 -- 172.21.15.55:0/171985931 <== mgr.14144 v2:172.21.15.55:6800/1826150465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fc640106a70 con 0x7fc62c033c20 2023-12-06T03:21:10.980 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled prometheus update... 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc62c033c20 msgr2=0x7fc62c036010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc62c033c20 0x7fc62c036010 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fc634006fd0 tx=0x7fc634006d30).stop 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 msgr2=0x7fc64010cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc64010cbf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fc63000b190 tx=0x7fc630006ee0).stop 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 shutdown_connections 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc62c033c20 0x7fc62c036010 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6400724b0 0x7fc64010cbf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 --2- 172.21.15.55:0/171985931 >> [v2:172.21.15.55:6800/3025569119,v1:172.21.15.55:6801/3025569119] conn(0x7fc62c02fe00 0x7fc62c0322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:21:10.982 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.979+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 >> 172.21.15.55:0/171985931 conn(0x7fc6400f74c0 msgr2=0x7fc6400f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:10.983 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.981+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 shutdown_connections 2023-12-06T03:21:10.983 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:10.981+0000 7fc64688a700 1 -- 172.21.15.55:0/171985931 wait complete. 2023-12-06T03:21:11.258 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:09.967816+0000 mon.smithi055 (mon.0) 66 : cluster [DBG] mgrmap e10: smithi055.qjkkyb(active, starting, since 0.317232s) 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:09.969135+0000 mon.smithi055 (mon.0) 67 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata", "id": "smithi055"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:09.969908+0000 mon.smithi055 (mon.0) 68 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mgr metadata", "who": "smithi055.qjkkyb", "id": "smithi055.qjkkyb"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:09.970794+0000 mon.smithi055 (mon.0) 69 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:09.971058+0000 mon.smithi055 (mon.0) 70 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:09.971327+0000 mon.smithi055 (mon.0) 71 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:09.985888+0000 mon.smithi055 (mon.0) 72 : cluster [INF] Manager daemon smithi055.qjkkyb is now available 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:10.106841+0000 mon.smithi055 (mon.0) 73 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:10.108093+0000 mon.smithi055 (mon.0) 74 : audit [DBG] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:10.125545+0000 mon.smithi055 (mon.0) 75 : audit [INF] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:21:11.259 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:10.128300+0000 mon.smithi055 (mon.0) 76 : audit [INF] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:21:11.330 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying grafana service with default placement... 2023-12-06T03:21:11.330 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply grafana 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7421d3700 1 Processor -- start 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7421d3700 1 -- start start 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7421d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c0fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7421d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0fb2e0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7411d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c0fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7411d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c0fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37900/0 (socket says 172.21.15.55:37900) 2023-12-06T03:21:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.727+0000 7ff7411d1700 1 -- 172.21.15.55:0/1386629027 learned_addr learned my addr 172.21.15.55:0/1386629027 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.728+0000 7ff7411d1700 1 -- 172.21.15.55:0/1386629027 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0fb420 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.728+0000 7ff7411d1700 1 --2- 172.21.15.55:0/1386629027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c0fadf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff73800b5f0 tx=0x7ff738006ee0).ready entity=mon.0 client_cookie=b783c7d0fb5a7ed8 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.728+0000 7ff72bfff700 1 -- 172.21.15.55:0/1386629027 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff73800d600 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.728+0000 7ff72bfff700 1 -- 172.21.15.55:0/1386629027 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff73800dbd0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.728+0000 7ff72bfff700 1 -- 172.21.15.55:0/1386629027 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff738011c00 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 -- 172.21.15.55:0/1386629027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 msgr2=0x7ff73c0fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 --2- 172.21.15.55:0/1386629027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c0fadf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff73800b5f0 tx=0x7ff738006ee0).stop 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 -- 172.21.15.55:0/1386629027 shutdown_connections 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 --2- 172.21.15.55:0/1386629027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c0fadf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 -- 172.21.15.55:0/1386629027 >> 172.21.15.55:0/1386629027 conn(0x7ff73c0f6210 msgr2=0x7ff73c0f8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 -- 172.21.15.55:0/1386629027 shutdown_connections 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.729+0000 7ff7421d3700 1 -- 172.21.15.55:0/1386629027 wait complete. 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.730+0000 7ff7421d3700 1 Processor -- start 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.730+0000 7ff7421d3700 1 -- start start 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7421d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7421d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c109bf0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7411d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7411d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c109700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37906/0 (socket says 172.21.15.55:37906) 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7411d1700 1 -- 172.21.15.55:0/2529066794 learned_addr learned my addr 172.21.15.55:0/2529066794 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7411d1700 1 -- 172.21.15.55:0/2529066794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73800b200 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7411d1700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c109700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff73800e040 tx=0x7ff738006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff738011e70 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff738022890 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c109df0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.731+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7380186e0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.732+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c10a200 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.733+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 11) v1 ==== 36054+0+0 (secure 0 0 0) 0x7ff738029060 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.733+0000 7ff729ffb700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7ff72402fcf0 0x7ff7240321b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.733+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7ff738042be0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.733+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c045ad0 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.733+0000 7ff7409d0700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7ff72402fcf0 0x7ff7240321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.733+0000 7ff7409d0700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7ff72402fcf0 0x7ff7240321b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff72c0096a0 tx=0x7ff72c006b40).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.738+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff738016070 con 0x7ff73c0fa9d0 2023-12-06T03:21:11.935 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.934+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 --> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7ff73c10ab40 con 0x7ff72402fcf0 2023-12-06T03:21:11.941 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.940+0000 7ff729ffb700 1 -- 172.21.15.55:0/2529066794 <== mgr.14144 v2:172.21.15.55:6800/1826150465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff73c10ab40 con 0x7ff72402fcf0 2023-12-06T03:21:11.941 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled grafana update... 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7ff72402fcf0 msgr2=0x7ff7240321b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7ff72402fcf0 0x7ff7240321b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff72c0096a0 tx=0x7ff72c006b40).stop 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 msgr2=0x7ff73c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c109700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff73800e040 tx=0x7ff738006ee0).stop 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 shutdown_connections 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7ff72402fcf0 0x7ff7240321b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 --2- 172.21.15.55:0/2529066794 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff73c0fa9d0 0x7ff73c109700 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 >> 172.21.15.55:0/2529066794 conn(0x7ff73c0f6210 msgr2=0x7ff73c0f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 shutdown_connections 2023-12-06T03:21:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:11.942+0000 7ff7421d3700 1 -- 172.21.15.55:0/2529066794 wait complete. 2023-12-06T03:21:12.225 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:10.971138+0000 mon.smithi055 (mon.0) 77 : cluster [DBG] mgrmap e11: smithi055.qjkkyb(active, since 1.32054s) 2023-12-06T03:21:12.225 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:10.972340+0000 mgr.smithi055.qjkkyb (mgr.14144) 2 : cephadm [INF] Saving service prometheus spec with placement count:1 2023-12-06T03:21:12.225 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:10.978015+0000 mon.smithi055 (mon.0) 78 : audit [INF] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:12.225 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:11.941133+0000 mon.smithi055 (mon.0) 79 : audit [INF] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying node-exporter service with default placement... 2023-12-06T03:21:12.289 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply node-exporter 2023-12-06T03:21:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb37ac700 1 Processor -- start 2023-12-06T03:21:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb37ac700 1 -- start start 2023-12-06T03:21:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb37ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac0fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:12.668 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb37ac700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0fedd0 con 0x7f4fac070430 2023-12-06T03:21:12.670 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb1548700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac0fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb1548700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac0fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37914/0 (socket says 172.21.15.55:37914) 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.666+0000 7f4fb1548700 1 -- 172.21.15.55:0/3190363729 learned_addr learned my addr 172.21.15.55:0/3190363729 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4fb1548700 1 -- 172.21.15.55:0/3190363729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac0fef10 con 0x7f4fac070430 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4fb1548700 1 --2- 172.21.15.55:0/3190363729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac0fe8e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f4fa800b5f0 tx=0x7f4fa8006ee0).ready entity=mon.0 client_cookie=d4b5de3b0d4dd8bb server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4f9bfff700 1 -- 172.21.15.55:0/3190363729 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa800d600 con 0x7f4fac070430 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4f9bfff700 1 -- 172.21.15.55:0/3190363729 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f4fa800dbd0 con 0x7f4fac070430 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4f9bfff700 1 -- 172.21.15.55:0/3190363729 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa8011c00 con 0x7f4fac070430 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4fb37ac700 1 -- 172.21.15.55:0/3190363729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 msgr2=0x7f4fac0fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.667+0000 7f4fb37ac700 1 --2- 172.21.15.55:0/3190363729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac0fe8e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f4fa800b5f0 tx=0x7f4fa8006ee0).stop 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 -- 172.21.15.55:0/3190363729 shutdown_connections 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 --2- 172.21.15.55:0/3190363729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac0fe8e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 -- 172.21.15.55:0/3190363729 >> 172.21.15.55:0/3190363729 conn(0x7f4fac0f63c0 msgr2=0x7f4fac0f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 -- 172.21.15.55:0/3190363729 shutdown_connections 2023-12-06T03:21:12.671 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 -- 172.21.15.55:0/3190363729 wait complete. 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 Processor -- start 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.668+0000 7f4fb37ac700 1 -- start start 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb37ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac10aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb37ac700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac10af30 con 0x7f4fac070430 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb1548700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac10aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb1548700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac10aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37930/0 (socket says 172.21.15.55:37930) 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb1548700 1 -- 172.21.15.55:0/222429291 learned_addr learned my addr 172.21.15.55:0/222429291 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb1548700 1 -- 172.21.15.55:0/222429291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa800b200 con 0x7f4fac070430 2023-12-06T03:21:12.672 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb1548700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac10aa40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f4fa800e040 tx=0x7f4fa8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa800d9f0 con 0x7f4fac070430 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f4fa801a9f0 con 0x7f4fac070430 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.669+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac10b130 con 0x7f4fac070430 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.670+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac10b540 con 0x7f4fac070430 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.670+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fa801b8c0 con 0x7f4fac070430 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.670+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f4fa800f070 con 0x7f4fac070430 2023-12-06T03:21:12.673 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.671+0000 7f4f99ffb700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f4f9402feb0 0x7f4f94032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.671+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f4fa801ab60 con 0x7f4fac070430 2023-12-06T03:21:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.671+0000 7f4fb0d47700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f4f9402feb0 0x7f4f94032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.671+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c004fa0 con 0x7f4fac070430 2023-12-06T03:21:12.674 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.671+0000 7f4fb0d47700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f4f9402feb0 0x7f4f94032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f4f9c0096a0 tx=0x7f4f9c006b40).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:12.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.676+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4fa8016030 con 0x7f4fac070430 2023-12-06T03:21:12.870 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.868+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 --> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f8c002a50 con 0x7f4f9402feb0 2023-12-06T03:21:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.873+0000 7f4f99ffb700 1 -- 172.21.15.55:0/222429291 <== mgr.14144 v2:172.21.15.55:6800/1826150465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f4f8c002a50 con 0x7f4f9402feb0 2023-12-06T03:21:12.874 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled node-exporter update... 2023-12-06T03:21:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f4f9402feb0 msgr2=0x7f4f94032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f4f9402feb0 0x7f4f94032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f4f9c0096a0 tx=0x7f4f9c006b40).stop 2023-12-06T03:21:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 msgr2=0x7f4fac10aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac10aa40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f4fa800e040 tx=0x7f4fa8006ee0).stop 2023-12-06T03:21:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 shutdown_connections 2023-12-06T03:21:12.877 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f4f9402feb0 0x7f4f94032370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 --2- 172.21.15.55:0/222429291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4fac070430 0x7f4fac10aa40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 >> 172.21.15.55:0/222429291 conn(0x7f4fac0f63c0 msgr2=0x7f4fac0f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 shutdown_connections 2023-12-06T03:21:12.878 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:12.875+0000 7f4fb37ac700 1 -- 172.21.15.55:0/222429291 wait complete. 2023-12-06T03:21:13.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:11.936028+0000 mgr.smithi055.qjkkyb (mgr.14144) 3 : audit [DBG] from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:13.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:11.936604+0000 mgr.smithi055.qjkkyb (mgr.14144) 4 : cephadm [INF] Saving service grafana spec with placement count:1 2023-12-06T03:21:13.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:11.979318+0000 mon.smithi055 (mon.0) 80 : cluster [DBG] mgrmap e12: smithi055.qjkkyb(active, since 2s) 2023-12-06T03:21:13.201 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:12.873989+0000 mon.smithi055 (mon.0) 81 : audit [INF] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:13.222 INFO:teuthology.orchestra.run.smithi055.stderr:Deploying alertmanager service with default placement... 2023-12-06T03:21:13.222 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply alertmanager 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f10766ba700 1 Processor -- start 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f10766ba700 1 -- start start 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f10766ba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f1070070a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f10766ba700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070072910 con 0x7f10700724f0 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f106ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f1070070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f106ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f1070070a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37936/0 (socket says 172.21.15.55:37936) 2023-12-06T03:21:13.588 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.584+0000 7f106ffff700 1 -- 172.21.15.55:0/511307180 learned_addr learned my addr 172.21.15.55:0/511307180 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.585+0000 7f106ffff700 1 -- 172.21.15.55:0/511307180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070070f80 con 0x7f10700724f0 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.585+0000 7f106ffff700 1 --2- 172.21.15.55:0/511307180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f1070070a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f106000b5f0 tx=0x7f1060006ee0).ready entity=mon.0 client_cookie=6dc07e1aba2dbbfb server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.585+0000 7f106effd700 1 -- 172.21.15.55:0/511307180 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f106000d600 con 0x7f10700724f0 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.585+0000 7f106effd700 1 -- 172.21.15.55:0/511307180 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f106000dbd0 con 0x7f10700724f0 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.585+0000 7f106effd700 1 -- 172.21.15.55:0/511307180 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1060011c00 con 0x7f10700724f0 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 -- 172.21.15.55:0/511307180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 msgr2=0x7f1070070a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 --2- 172.21.15.55:0/511307180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f1070070a90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f106000b5f0 tx=0x7f1060006ee0).stop 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 -- 172.21.15.55:0/511307180 shutdown_connections 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 --2- 172.21.15.55:0/511307180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f1070070a90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 -- 172.21.15.55:0/511307180 >> 172.21.15.55:0/511307180 conn(0x7f10700f74c0 msgr2=0x7f10700f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 -- 172.21.15.55:0/511307180 shutdown_connections 2023-12-06T03:21:13.589 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.586+0000 7f10766ba700 1 -- 172.21.15.55:0/511307180 wait complete. 2023-12-06T03:21:13.590 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f10766ba700 1 Processor -- start 2023-12-06T03:21:13.591 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f10766ba700 1 -- start start 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f10766ba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f107010ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f10766ba700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107010d1e0 con 0x7f10700724f0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f106ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f107010ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f106ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f107010ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37940/0 (socket says 172.21.15.55:37940) 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.587+0000 7f106ffff700 1 -- 172.21.15.55:0/2470581201 learned_addr learned my addr 172.21.15.55:0/2470581201 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f106ffff700 1 -- 172.21.15.55:0/2470581201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106000b200 con 0x7f10700724f0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f106ffff700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f107010ccf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1060006c70 tx=0x7f1060006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f106000da30 con 0x7f10700724f0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f106001a9f0 con 0x7f10700724f0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107010d3e0 con 0x7f10700724f0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f106001b890 con 0x7f10700724f0 2023-12-06T03:21:13.592 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.588+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107010d7f0 con 0x7f10700724f0 2023-12-06T03:21:13.593 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.589+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f106000f070 con 0x7f10700724f0 2023-12-06T03:21:13.593 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.589+0000 7f106cff9700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f105c02feb0 0x7f105c032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:13.593 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.589+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070045ad0 con 0x7f10700724f0 2023-12-06T03:21:13.593 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.589+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f10600413e0 con 0x7f10700724f0 2023-12-06T03:21:13.593 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.589+0000 7f106f7fe700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f105c02feb0 0x7f105c032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:13.593 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.590+0000 7f106f7fe700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f105c02feb0 0x7f105c032370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f10640096a0 tx=0x7f1064006b40).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:13.595 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.594+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1060016020 con 0x7f10700724f0 2023-12-06T03:21:13.791 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.789+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 --> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f107010df20 con 0x7f105c02feb0 2023-12-06T03:21:13.795 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.793+0000 7f106cff9700 1 -- 172.21.15.55:0/2470581201 <== mgr.14144 v2:172.21.15.55:6800/1826150465 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f107010df20 con 0x7f105c02feb0 2023-12-06T03:21:13.795 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Scheduled alertmanager update... 2023-12-06T03:21:13.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f105c02feb0 msgr2=0x7f105c032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:13.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f105c02feb0 0x7f105c032370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f10640096a0 tx=0x7f1064006b40).stop 2023-12-06T03:21:13.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 msgr2=0x7f107010ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:13.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f107010ccf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1060006c70 tx=0x7f1060006ee0).stop 2023-12-06T03:21:13.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 shutdown_connections 2023-12-06T03:21:13.798 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f105c02feb0 0x7f105c032370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:13.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 --2- 172.21.15.55:0/2470581201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10700724f0 0x7f107010ccf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:13.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.795+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 >> 172.21.15.55:0/2470581201 conn(0x7f10700f74c0 msgr2=0x7f10700f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:13.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.796+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 shutdown_connections 2023-12-06T03:21:13.799 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:13.796+0000 7f10766ba700 1 -- 172.21.15.55:0/2470581201 wait complete. 2023-12-06T03:21:14.083 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:12.870607+0000 mgr.smithi055.qjkkyb (mgr.14144) 5 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:14.083 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:12.871319+0000 mgr.smithi055.qjkkyb (mgr.14144) 6 : cephadm [INF] Saving service node-exporter spec with placement * 2023-12-06T03:21:14.083 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:13.794354+0000 mon.smithi055 (mon.0) 82 : audit [INF] from='mgr.14144 172.21.15.55:0/2719257320' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/cephadm/container_init True --force 2023-12-06T03:21:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5ec1f700 1 Processor -- start 2023-12-06T03:21:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5ec1f700 1 -- start start 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5ec1f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce580fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5ec1f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce58070790 con 0x7fce58070370 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5c9bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce580fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5c9bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce580fe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37948/0 (socket says 172.21.15.55:37948) 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.532+0000 7fce5c9bb700 1 -- 172.21.15.55:0/2756871736 learned_addr learned my addr 172.21.15.55:0/2756871736 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce5c9bb700 1 -- 172.21.15.55:0/2756871736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580fee90 con 0x7fce58070370 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce5c9bb700 1 --2- 172.21.15.55:0/2756871736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce580fe9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fce4c00b5f0 tx=0x7fce4c006ee0).ready entity=mon.0 client_cookie=18014f091bd2cde0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce577fe700 1 -- 172.21.15.55:0/2756871736 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce4c00d600 con 0x7fce58070370 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce577fe700 1 -- 172.21.15.55:0/2756871736 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fce4c00dbd0 con 0x7fce58070370 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce577fe700 1 -- 172.21.15.55:0/2756871736 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce4c011c00 con 0x7fce58070370 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce5ec1f700 1 -- 172.21.15.55:0/2756871736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 msgr2=0x7fce580fe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.533+0000 7fce5ec1f700 1 --2- 172.21.15.55:0/2756871736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce580fe9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fce4c00b5f0 tx=0x7fce4c006ee0).stop 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 -- 172.21.15.55:0/2756871736 shutdown_connections 2023-12-06T03:21:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 --2- 172.21.15.55:0/2756871736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce580fe9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 -- 172.21.15.55:0/2756871736 >> 172.21.15.55:0/2756871736 conn(0x7fce580f63c0 msgr2=0x7fce580f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 -- 172.21.15.55:0/2756871736 shutdown_connections 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 -- 172.21.15.55:0/2756871736 wait complete. 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 Processor -- start 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.534+0000 7fce5ec1f700 1 -- start start 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5ec1f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce5810a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5ec1f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5810aee0 con 0x7fce58070370 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5c9bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce5810a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5c9bb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce5810a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37956/0 (socket says 172.21.15.55:37956) 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5c9bb700 1 -- 172.21.15.55:0/3344366821 learned_addr learned my addr 172.21.15.55:0/3344366821 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5c9bb700 1 -- 172.21.15.55:0/3344366821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c00b200 con 0x7fce58070370 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5c9bb700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce5810a9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fce4c00e040 tx=0x7fce4c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce4c00d9f0 con 0x7fce58070370 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fce4c01a9f0 con 0x7fce58070370 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5810b0e0 con 0x7fce58070370 2023-12-06T03:21:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce4c01b890 con 0x7fce58070370 2023-12-06T03:21:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.535+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5810b4f0 con 0x7fce58070370 2023-12-06T03:21:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.536+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7fce4c00f070 con 0x7fce58070370 2023-12-06T03:21:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.536+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce58045ad0 con 0x7fce58070370 2023-12-06T03:21:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.537+0000 7fce557fa700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fce3c02feb0 0x7fce3c032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.537+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fce4c0414e0 con 0x7fce58070370 2023-12-06T03:21:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.537+0000 7fce57fff700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fce3c02feb0 0x7fce3c032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.537+0000 7fce57fff700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fce3c02feb0 0x7fce3c032370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fce440096a0 tx=0x7fce44006b40).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.541+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fce4c022d70 con 0x7fce58070370 2023-12-06T03:21:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.705+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fce580ffb00 con 0x7fce58070370 2023-12-06T03:21:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.710+0000 7fce557fa700 1 -- 172.21.15.55:0/3344366821 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v8)=0 v8) v1 ==== 142+0+0 (secure 0 0 0) 0x7fce4c0160d0 con 0x7fce58070370 2023-12-06T03:21:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.712+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fce3c02feb0 msgr2=0x7fce3c032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.712+0000 7fce5ec1f700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fce3c02feb0 0x7fce3c032370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fce440096a0 tx=0x7fce44006b40).stop 2023-12-06T03:21:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.712+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 msgr2=0x7fce5810a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.712+0000 7fce5ec1f700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce5810a9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fce4c00e040 tx=0x7fce4c006ee0).stop 2023-12-06T03:21:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.713+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 shutdown_connections 2023-12-06T03:21:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.713+0000 7fce5ec1f700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fce3c02feb0 0x7fce3c032370 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:14.717 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.713+0000 7fce5ec1f700 1 --2- 172.21.15.55:0/3344366821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce58070370 0x7fce5810a9f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:14.717 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.713+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 >> 172.21.15.55:0/3344366821 conn(0x7fce580f63c0 msgr2=0x7fce580f7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:14.717 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.713+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 shutdown_connections 2023-12-06T03:21:14.717 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:14.713+0000 7fce5ec1f700 1 -- 172.21.15.55:0/3344366821 wait complete. 2023-12-06T03:21:15.008 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:13.791135+0000 mgr.smithi055.qjkkyb (mgr.14144) 7 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:15.008 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:13.791729+0000 mgr.smithi055.qjkkyb (mgr.14144) 8 : cephadm [INF] Saving service alertmanager spec with placement count:1 2023-12-06T03:21:15.008 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:14.711919+0000 mon.smithi055 (mon.0) 83 : audit [INF] from='client.? 172.21.15.55:0/3344366821' entity='client.admin' 2023-12-06T03:21:15.072 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/dashboard/ssl_server_port 8443 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.456+0000 7f8d13fff700 1 Processor -- start 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.456+0000 7f8d13fff700 1 -- start start 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.456+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d14100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.456+0000 7f8d13fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140fc5a0 con 0x7f8d140703b0 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d14100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d14100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37958/0 (socket says 172.21.15.55:37958) 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d12ffd700 1 -- 172.21.15.55:0/4283058924 learned_addr learned my addr 172.21.15.55:0/4283058924 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:15.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d12ffd700 1 -- 172.21.15.55:0/4283058924 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d141010a0 con 0x7f8d140703b0 2023-12-06T03:21:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d12ffd700 1 --2- 172.21.15.55:0/4283058924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d14100bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8d0400b5f0 tx=0x7f8d04006ee0).ready entity=mon.0 client_cookie=e4e92c9efd3fffe7 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d11ffb700 1 -- 172.21.15.55:0/4283058924 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d0400d600 con 0x7f8d140703b0 2023-12-06T03:21:15.460 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d11ffb700 1 -- 172.21.15.55:0/4283058924 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8d0400dbd0 con 0x7f8d140703b0 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.457+0000 7f8d11ffb700 1 -- 172.21.15.55:0/4283058924 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d04011c00 con 0x7f8d140703b0 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 -- 172.21.15.55:0/4283058924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 msgr2=0x7f8d14100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 --2- 172.21.15.55:0/4283058924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d14100bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f8d0400b5f0 tx=0x7f8d04006ee0).stop 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 -- 172.21.15.55:0/4283058924 shutdown_connections 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 --2- 172.21.15.55:0/4283058924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d14100bb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 -- 172.21.15.55:0/4283058924 >> 172.21.15.55:0/4283058924 conn(0x7f8d140f6210 msgr2=0x7f8d140f8630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 -- 172.21.15.55:0/4283058924 shutdown_connections 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.458+0000 7f8d13fff700 1 -- 172.21.15.55:0/4283058924 wait complete. 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d13fff700 1 Processor -- start 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d13fff700 1 -- start start 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d1410ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:15.461 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d13fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1410cf60 con 0x7f8d140703b0 2023-12-06T03:21:15.462 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d1410ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d1410ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37974/0 (socket says 172.21.15.55:37974) 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d12ffd700 1 -- 172.21.15.55:0/2634771409 learned_addr learned my addr 172.21.15.55:0/2634771409 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.459+0000 7f8d12ffd700 1 -- 172.21.15.55:0/2634771409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0400b200 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.460+0000 7f8d12ffd700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d1410ca70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8d0400e040 tx=0x7f8d04006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.460+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d0400d9f0 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.460+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8d0401a9f0 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.460+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1410d160 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.460+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d0401b8c0 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.460+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1410d570 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.461+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f8d0400f070 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.461+0000 7f8d0b7fe700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f8cf402fe60 0x7f8cf4032320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.461+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f8d04041450 con 0x7f8d140703b0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.461+0000 7f8d127fc700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f8cf402fe60 0x7f8cf4032320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.461+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d14045ad0 con 0x7f8d140703b0 2023-12-06T03:21:15.464 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.461+0000 7f8d127fc700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f8cf402fe60 0x7f8cf4032320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f8cfc0096a0 tx=0x7f8cfc006b40).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:15.467 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.466+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8d04016070 con 0x7f8d140703b0 2023-12-06T03:21:15.632 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.630+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f8d141068f0 con 0x7f8d140703b0 2023-12-06T03:21:15.637 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.636+0000 7f8d0b7fe700 1 -- 172.21.15.55:0/2634771409 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v9)=0 v9) v1 ==== 130+0+0 (secure 0 0 0) 0x7f8d04025460 con 0x7f8d140703b0 2023-12-06T03:21:15.640 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f8cf402fe60 msgr2=0x7f8cf4032320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f8cf402fe60 0x7f8cf4032320 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f8cfc0096a0 tx=0x7f8cfc006b40).stop 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 msgr2=0x7f8d1410ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d1410ca70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8d0400e040 tx=0x7f8d04006ee0).stop 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 shutdown_connections 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f8cf402fe60 0x7f8cf4032320 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 --2- 172.21.15.55:0/2634771409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d140703b0 0x7f8d1410ca70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:15.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 >> 172.21.15.55:0/2634771409 conn(0x7f8d140f6210 msgr2=0x7f8d140f7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:15.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 shutdown_connections 2023-12-06T03:21:15.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:15.638+0000 7f8d13fff700 1 -- 172.21.15.55:0/2634771409 wait complete. 2023-12-06T03:21:15.980 INFO:teuthology.orchestra.run.smithi055.stderr:Enabling the dashboard module... 2023-12-06T03:21:15.980 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable dashboard 2023-12-06T03:21:16.359 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13f1e37700 1 Processor -- start 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13f1e37700 1 -- start start 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13f1e37700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13f1e37700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0fede0 con 0x7f13ec0704a0 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37978/0 (socket says 172.21.15.55:37978) 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13eb7fe700 1 -- 172.21.15.55:0/2068626261 learned_addr learned my addr 172.21.15.55:0/2068626261 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.357+0000 7f13eb7fe700 1 -- 172.21.15.55:0/2068626261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0fef20 con 0x7f13ec0704a0 2023-12-06T03:21:16.360 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13eb7fe700 1 --2- 172.21.15.55:0/2068626261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fe8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f13dc00b5f0 tx=0x7f13dc006ee0).ready entity=mon.0 client_cookie=13d0f664a87cfc87 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:16.362 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13ea7fc700 1 -- 172.21.15.55:0/2068626261 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13dc00d600 con 0x7f13ec0704a0 2023-12-06T03:21:16.362 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13ea7fc700 1 -- 172.21.15.55:0/2068626261 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f13dc00dbd0 con 0x7f13ec0704a0 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13ea7fc700 1 -- 172.21.15.55:0/2068626261 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13dc011c00 con 0x7f13ec0704a0 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13f1e37700 1 -- 172.21.15.55:0/2068626261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 msgr2=0x7f13ec0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13f1e37700 1 --2- 172.21.15.55:0/2068626261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fe8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f13dc00b5f0 tx=0x7f13dc006ee0).stop 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13f1e37700 1 -- 172.21.15.55:0/2068626261 shutdown_connections 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13f1e37700 1 --2- 172.21.15.55:0/2068626261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fe8f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.358+0000 7f13f1e37700 1 -- 172.21.15.55:0/2068626261 >> 172.21.15.55:0/2068626261 conn(0x7f13ec0f63b0 msgr2=0x7f13ec0f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.359+0000 7f13f1e37700 1 -- 172.21.15.55:0/2068626261 shutdown_connections 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.359+0000 7f13f1e37700 1 -- 172.21.15.55:0/2068626261 wait complete. 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.359+0000 7f13f1e37700 1 Processor -- start 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.359+0000 7f13f1e37700 1 -- start start 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13f1e37700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13f1e37700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0fa7f0 con 0x7f13ec0704a0 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37984/0 (socket says 172.21.15.55:37984) 2023-12-06T03:21:16.363 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13eb7fe700 1 -- 172.21.15.55:0/2700857354 learned_addr learned my addr 172.21.15.55:0/2700857354 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13eb7fe700 1 -- 172.21.15.55:0/2700857354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc00b200 con 0x7f13ec0704a0 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13eb7fe700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fc370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f13dc00b190 tx=0x7f13dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.360+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13dc01b410 con 0x7f13ec0704a0 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.361+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f13dc01b9e0 con 0x7f13ec0704a0 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.361+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec0fa990 con 0x7f13ec0704a0 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.361+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f13dc01a880 con 0x7f13ec0704a0 2023-12-06T03:21:16.364 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.361+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec0fada0 con 0x7f13ec0704a0 2023-12-06T03:21:16.365 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.362+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec045ad0 con 0x7f13ec0704a0 2023-12-06T03:21:16.365 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.362+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f13dc00f070 con 0x7f13ec0704a0 2023-12-06T03:21:16.365 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.362+0000 7f13d3fff700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f13d802feb0 0x7f13d8032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:16.365 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.362+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f13dc0413b0 con 0x7f13ec0704a0 2023-12-06T03:21:16.365 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.362+0000 7f13eaffd700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f13d802feb0 0x7f13d8032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:16.365 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.363+0000 7f13eaffd700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f13d802feb0 0x7f13d8032370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f13e00096a0 tx=0x7f13e0006b40).ready entity=mgr.14144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:16.368 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.366+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f13dc019950 con 0x7f13ec0704a0 2023-12-06T03:21:16.574 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.572+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f13ec0ffcb0 con 0x7f13ec0704a0 2023-12-06T03:21:16.638 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.636+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f13dc01bb50 con 0x7f13ec0704a0 2023-12-06T03:21:16.639 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.637+0000 7f13d3fff700 1 -- 172.21.15.55:0/2700857354 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v13) v1 ==== 88+0+0 (secure 0 0 0) 0x7f13dc005d10 con 0x7f13ec0704a0 2023-12-06T03:21:16.641 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.639+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f13d802feb0 msgr2=0x7f13d8032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.639+0000 7f13f1e37700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f13d802feb0 0x7f13d8032370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f13e00096a0 tx=0x7f13e0006b40).stop 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.639+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 msgr2=0x7f13ec0fc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.639+0000 7f13f1e37700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fc370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f13dc00b190 tx=0x7f13dc006ee0).stop 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.640+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 shutdown_connections 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.640+0000 7f13f1e37700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7f13d802feb0 0x7f13d8032370 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.640+0000 7f13f1e37700 1 --2- 172.21.15.55:0/2700857354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13ec0704a0 0x7f13ec0fc370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.640+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 >> 172.21.15.55:0/2700857354 conn(0x7f13ec0f63b0 msgr2=0x7f13ec0f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.641+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 shutdown_connections 2023-12-06T03:21:16.643 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:16.641+0000 7f13f1e37700 1 -- 172.21.15.55:0/2700857354 wait complete. 2023-12-06T03:21:16.932 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:16 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:15.637058+0000 mon.smithi055 (mon.0) 84 : audit [INF] from='client.? 172.21.15.55:0/2634771409' entity='client.admin' 2023-12-06T03:21:16.932 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:16 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:16.573863+0000 mon.smithi055 (mon.0) 85 : audit [INF] from='client.? 172.21.15.55:0/2700857354' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-06T03:21:17.005 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2023-12-06T03:21:17.445 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.443+0000 7fca0c4a1700 1 Processor -- start 2023-12-06T03:21:17.445 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.443+0000 7fca0c4a1700 1 -- start start 2023-12-06T03:21:17.446 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.443+0000 7fca0c4a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca040fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:17.446 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.443+0000 7fca0c4a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040fb490 con 0x7fca040fab80 2023-12-06T03:21:17.446 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.444+0000 7fca0a23d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca040fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:17.446 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.444+0000 7fca0a23d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca040fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38014/0 (socket says 172.21.15.55:38014) 2023-12-06T03:21:17.446 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.444+0000 7fca0a23d700 1 -- 172.21.15.55:0/2594696321 learned_addr learned my addr 172.21.15.55:0/2594696321 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:17.448 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.446+0000 7fca0a23d700 1 -- 172.21.15.55:0/2594696321 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040fb5d0 con 0x7fca040fab80 2023-12-06T03:21:17.448 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.446+0000 7fca0a23d700 1 --2- 172.21.15.55:0/2594696321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca040fafa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fca0000b870 tx=0x7fca00006f70).ready entity=mon.0 client_cookie=6e75022ccdf1618e server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:17.448 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.446+0000 7fca0923b700 1 -- 172.21.15.55:0/2594696321 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca0000da20 con 0x7fca040fab80 2023-12-06T03:21:17.448 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.446+0000 7fca0923b700 1 -- 172.21.15.55:0/2594696321 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fca00010470 con 0x7fca040fab80 2023-12-06T03:21:17.448 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.446+0000 7fca0923b700 1 -- 172.21.15.55:0/2594696321 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca0000f5b0 con 0x7fca040fab80 2023-12-06T03:21:17.451 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 -- 172.21.15.55:0/2594696321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 msgr2=0x7fca040fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:17.451 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 --2- 172.21.15.55:0/2594696321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca040fafa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fca0000b870 tx=0x7fca00006f70).stop 2023-12-06T03:21:17.451 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 -- 172.21.15.55:0/2594696321 shutdown_connections 2023-12-06T03:21:17.451 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 --2- 172.21.15.55:0/2594696321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca040fafa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 -- 172.21.15.55:0/2594696321 >> 172.21.15.55:0/2594696321 conn(0x7fca040f63c0 msgr2=0x7fca040f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 -- 172.21.15.55:0/2594696321 shutdown_connections 2023-12-06T03:21:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.449+0000 7fca0c4a1700 1 -- 172.21.15.55:0/2594696321 wait complete. 2023-12-06T03:21:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.450+0000 7fca0c4a1700 1 Processor -- start 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.450+0000 7fca0c4a1700 1 -- start start 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0c4a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca04108770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0c4a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04108c60 con 0x7fca040fab80 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0a23d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca04108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0a23d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca04108770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38022/0 (socket says 172.21.15.55:38022) 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0a23d700 1 -- 172.21.15.55:0/678205013 learned_addr learned my addr 172.21.15.55:0/678205013 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0a23d700 1 -- 172.21.15.55:0/678205013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0000b480 con 0x7fca040fab80 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.451+0000 7fca0a23d700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca04108770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fca00006800 tx=0x7fca00006f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.452+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca0000f820 con 0x7fca040fab80 2023-12-06T03:21:17.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.452+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca04108e60 con 0x7fca040fab80 2023-12-06T03:21:17.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.452+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fca0000fdf0 con 0x7fca040fab80 2023-12-06T03:21:17.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.452+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca00019d50 con 0x7fca040fab80 2023-12-06T03:21:17.454 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.452+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca04109270 con 0x7fca040fab80 2023-12-06T03:21:17.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.453+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fca00016030 con 0x7fca040fab80 2023-12-06T03:21:17.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.453+0000 7fc9f2ffd700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 0x7fc9ec032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:17.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.453+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fca000442a0 con 0x7fca040fab80 2023-12-06T03:21:17.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.453+0000 7fca09a3c700 1 -- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 msgr2=0x7fc9ec032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:17.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.453+0000 7fca09a3c700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 0x7fc9ec032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-12-06T03:21:17.455 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.453+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca04045ad0 con 0x7fca040fab80 2023-12-06T03:21:17.459 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.458+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fca000189a0 con 0x7fca040fab80 2023-12-06T03:21:17.655 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.654+0000 7fca09a3c700 1 -- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 msgr2=0x7fc9ec032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:17.655 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.654+0000 7fca09a3c700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 0x7fc9ec032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.674+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fca041026a0 con 0x7fca040fab80 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.675+0000 7fc9f2ffd700 1 -- 172.21.15.55:0/678205013 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v13) v1 ==== 56+0+104 (secure 0 0 0) 0x7fca00014030 con 0x7fca040fab80 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "epoch": 13, 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "available": true, 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "active_name": "smithi055.qjkkyb", 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "num_standby": 0 2023-12-06T03:21:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 msgr2=0x7fc9ec032370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 0x7fc9ec032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 msgr2=0x7fca04108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca04108770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fca00006800 tx=0x7fca00006f70).stop 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 shutdown_connections 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fc9ec02feb0 0x7fc9ec032370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:21:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 --2- 172.21.15.55:0/678205013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca040fab80 0x7fca04108770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:17.680 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.677+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 >> 172.21.15.55:0/678205013 conn(0x7fca040f63c0 msgr2=0x7fca040ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:17.680 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.678+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 shutdown_connections 2023-12-06T03:21:17.680 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:17.678+0000 7fca0c4a1700 1 -- 172.21.15.55:0/678205013 wait complete. 2023-12-06T03:21:17.820 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:16.638875+0000 mon.smithi055 (mon.0) 86 : audit [INF] from='client.? 172.21.15.55:0/2700857354' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-06T03:21:17.820 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:16.638970+0000 mon.smithi055 (mon.0) 87 : cluster [DBG] mgrmap e13: smithi055.qjkkyb(active, since 6s) 2023-12-06T03:21:18.021 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for the mgr to restart... 2023-12-06T03:21:18.021 INFO:teuthology.orchestra.run.smithi055.stderr:Waiting for mgr epoch 13... 2023-12-06T03:21:18.021 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2023-12-06T03:21:18.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:18 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:17.676328+0000 mon.smithi055 (mon.0) 88 : audit [DBG] from='client.? 172.21.15.55:0/678205013' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a46bc700 1 Processor -- start 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a46bc700 1 -- start start 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a46bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c070a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a46bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0728f0 con 0x7fb59c0724d0 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a2458700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a2458700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c070a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38024/0 (socket says 172.21.15.55:38024) 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.530+0000 7fb5a2458700 1 -- 172.21.15.55:0/1295646061 learned_addr learned my addr 172.21.15.55:0/1295646061 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.531+0000 7fb5a2458700 1 -- 172.21.15.55:0/1295646061 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c070f60 con 0x7fb59c0724d0 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.531+0000 7fb5a2458700 1 --2- 172.21.15.55:0/1295646061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c070a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb59800b5f0 tx=0x7fb598006ee0).ready entity=mon.0 client_cookie=b38d4a55b3e4443a server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.531+0000 7fb5a1456700 1 -- 172.21.15.55:0/1295646061 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb59800d600 con 0x7fb59c0724d0 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.531+0000 7fb5a1456700 1 -- 172.21.15.55:0/1295646061 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fb59800dbd0 con 0x7fb59c0724d0 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.531+0000 7fb5a1456700 1 -- 172.21.15.55:0/1295646061 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb598011c00 con 0x7fb59c0724d0 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1295646061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 msgr2=0x7fb59c070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1295646061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c070a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb59800b5f0 tx=0x7fb598006ee0).stop 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1295646061 shutdown_connections 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1295646061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c070a70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1295646061 >> 172.21.15.55:0/1295646061 conn(0x7fb59c0f74c0 msgr2=0x7fb59c0f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1295646061 shutdown_connections 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1295646061 wait complete. 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 Processor -- start 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.532+0000 7fb5a46bc700 1 -- start start 2023-12-06T03:21:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a46bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c10cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a46bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c10d220 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a2458700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c10cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a2458700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c10cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38028/0 (socket says 172.21.15.55:38028) 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a2458700 1 -- 172.21.15.55:0/1849885441 learned_addr learned my addr 172.21.15.55:0/1849885441 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a2458700 1 -- 172.21.15.55:0/1849885441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59800b200 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a2458700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c10cd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb598006c70 tx=0x7fb598006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb59800da30 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fb598011690 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c10d420 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb59801b7b0 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.533+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c10d830 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.534+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fb59800f070 con 0x7fb59c0724d0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.534+0000 7fb58affd700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.535+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 --> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb584032a30 con 0x7fb58402feb0 2023-12-06T03:21:18.715 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.535+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fb5980412e0 con 0x7fb59c0724d0 2023-12-06T03:21:18.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.535+0000 7fb5a1c57700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 msgr2=0x7fb584032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:18.716 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.535+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-12-06T03:21:18.738 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.736+0000 7fb5a1c57700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 msgr2=0x7fb584032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:18.738 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:18.736+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-12-06T03:21:19.138 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:19.137+0000 7fb5a1c57700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 msgr2=0x7fb584032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:19.137+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-12-06T03:21:19.939 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:19.938+0000 7fb5a1c57700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 msgr2=0x7fb584032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:19.940 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:19.938+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-12-06T03:21:21.541 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:21.539+0000 7fb5a1c57700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 msgr2=0x7fb584032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/1826150465 2023-12-06T03:21:21.541 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:21.539+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-12-06T03:21:22.399 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:22.396+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mgrmap(e 14) v1 ==== 35997+0+0 (secure 0 0 0) 0x7fb59801b910 con 0x7fb59c0724d0 2023-12-06T03:21:22.399 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:22.396+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 msgr2=0x7fb584032370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-06T03:21:22.399 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:22.396+0000 7fb58affd700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:21:22.654 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:22.345355+0000 mon.smithi055 (mon.0) 89 : cluster [INF] Active manager daemon smithi055.qjkkyb restarted 2023-12-06T03:21:22.654 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:22.345912+0000 mon.smithi055 (mon.0) 90 : cluster [INF] Activating manager daemon smithi055.qjkkyb 2023-12-06T03:21:22.654 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:22.349470+0000 mon.smithi055 (mon.0) 91 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2023-12-06T03:21:23.405 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.403+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 15) v1 ==== 36067+0+0 (secure 0 0 0) 0x7fb5980254b0 con 0x7fb59c0724d0 2023-12-06T03:21:23.406 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.403+0000 7fb58affd700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb584033b40 0x7fb584035f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:23.406 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.403+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb584032a30 con 0x7fb584033b40 2023-12-06T03:21:23.406 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.404+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb584033b40 0x7fb584035f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:23.409 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.404+0000 7fb5a1c57700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb584033b40 0x7fb584035f30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fb58c006fd0 tx=0x7fb58c006cf0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:23.410 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.405+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mgr.14164 v2:172.21.15.55:6800/785191101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fb584032a30 con 0x7fb584033b40 2023-12-06T03:21:23.413 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.412+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb59c059100 con 0x7fb584033b40 2023-12-06T03:21:23.414 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.412+0000 7fb58affd700 1 -- 172.21.15.55:0/1849885441 <== mgr.14164 v2:172.21.15.55:6800/785191101 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fb59c059100 con 0x7fb584033b40 2023-12-06T03:21:23.414 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: { 2023-12-06T03:21:23.414 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "mgrmap_epoch": 15, 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: "initialized": true 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: } 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb584033b40 msgr2=0x7fb584035f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb584033b40 0x7fb584035f30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fb58c006fd0 tx=0x7fb58c006cf0).stop 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 msgr2=0x7fb59c10cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c10cd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb598006c70 tx=0x7fb598006ee0).stop 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 shutdown_connections 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb584033b40 0x7fb584035f30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb59c0724d0 0x7fb59c10cd30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 --2- 172.21.15.55:0/1849885441 >> [v2:172.21.15.55:6800/1826150465,v1:172.21.15.55:6801/1826150465] conn(0x7fb58402feb0 0x7fb584032370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 >> 172.21.15.55:0/1849885441 conn(0x7fb59c0f74c0 msgr2=0x7fb59c0f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 shutdown_connections 2023-12-06T03:21:23.416 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:23.413+0000 7fb5a46bc700 1 -- 172.21.15.55:0/1849885441 wait complete. 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:22.398957+0000 mon.smithi055 (mon.0) 92 : cluster [DBG] mgrmap e14: smithi055.qjkkyb(active, starting, since 0.0531377s) 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.402028+0000 mon.smithi055 (mon.0) 93 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata", "id": "smithi055"}]: dispatch 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.402730+0000 mon.smithi055 (mon.0) 94 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mgr metadata", "who": "smithi055.qjkkyb", "id": "smithi055.qjkkyb"}]: dispatch 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.403498+0000 mon.smithi055 (mon.0) 95 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.403690+0000 mon.smithi055 (mon.0) 96 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.403858+0000 mon.smithi055 (mon.0) 97 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:22.405936+0000 mon.smithi055 (mon.0) 98 : cluster [INF] Manager daemon smithi055.qjkkyb is now available 2023-12-06T03:21:23.683 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.518835+0000 mon.smithi055 (mon.0) 99 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:21:23.684 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.521358+0000 mon.smithi055 (mon.0) 100 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-06T03:21:23.684 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.546571+0000 mon.smithi055 (mon.0) 101 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:21:23.684 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:22.550797+0000 mon.smithi055 (mon.0) 102 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:21:23.804 INFO:teuthology.orchestra.run.smithi055.stderr:mgr epoch 13 is available 2023-12-06T03:21:23.804 INFO:teuthology.orchestra.run.smithi055.stderr:Generating a dashboard self-signed certificate... 2023-12-06T03:21:23.804 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 dashboard create-self-signed-cert 2023-12-06T03:21:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde4e5cf700 1 Processor -- start 2023-12-06T03:21:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde4e5cf700 1 -- start start 2023-12-06T03:21:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde4e5cf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde4e5cf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480fedd0 con 0x7fde48070430 2023-12-06T03:21:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde47fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde47fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39082/0 (socket says 172.21.15.55:39082) 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.204+0000 7fde47fff700 1 -- 172.21.15.55:0/1757107047 learned_addr learned my addr 172.21.15.55:0/1757107047 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.205+0000 7fde47fff700 1 -- 172.21.15.55:0/1757107047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480fef10 con 0x7fde48070430 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.205+0000 7fde47fff700 1 --2- 172.21.15.55:0/1757107047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fe8e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fde3800b5f0 tx=0x7fde38006ee0).ready entity=mon.0 client_cookie=d8023a4c147d2c07 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.205+0000 7fde46ffd700 1 -- 172.21.15.55:0/1757107047 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3800d600 con 0x7fde48070430 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.205+0000 7fde46ffd700 1 -- 172.21.15.55:0/1757107047 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fde3800dbd0 con 0x7fde48070430 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.205+0000 7fde46ffd700 1 -- 172.21.15.55:0/1757107047 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde38011c00 con 0x7fde48070430 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.206+0000 7fde4e5cf700 1 -- 172.21.15.55:0/1757107047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 msgr2=0x7fde480fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.206+0000 7fde4e5cf700 1 --2- 172.21.15.55:0/1757107047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fe8e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fde3800b5f0 tx=0x7fde38006ee0).stop 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.207+0000 7fde4e5cf700 1 -- 172.21.15.55:0/1757107047 shutdown_connections 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.207+0000 7fde4e5cf700 1 --2- 172.21.15.55:0/1757107047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fe8e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.207+0000 7fde4e5cf700 1 -- 172.21.15.55:0/1757107047 >> 172.21.15.55:0/1757107047 conn(0x7fde480f63c0 msgr2=0x7fde480f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.207+0000 7fde4e5cf700 1 -- 172.21.15.55:0/1757107047 shutdown_connections 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.207+0000 7fde4e5cf700 1 -- 172.21.15.55:0/1757107047 wait complete. 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.207+0000 7fde4e5cf700 1 Processor -- start 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde4e5cf700 1 -- start start 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde4e5cf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde4e5cf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480fa7e0 con 0x7fde48070430 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde47fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde47fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39084/0 (socket says 172.21.15.55:39084) 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde47fff700 1 -- 172.21.15.55:0/242198420 learned_addr learned my addr 172.21.15.55:0/242198420 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde47fff700 1 -- 172.21.15.55:0/242198420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3800b200 con 0x7fde48070430 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.208+0000 7fde47fff700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fc360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fde3800e040 tx=0x7fde38006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.209+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3800d9f0 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.209+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fde3801a9f0 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.209+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde480fa980 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.209+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3801b8c0 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.209+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde480fad90 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.210+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 15) v1 ==== 36067+0+0 (secure 0 0 0) 0x7fde3800f070 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.210+0000 7fde44ff9700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fde3402fd70 0x7fde34032230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.210+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fde38041400 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.210+0000 7fde477fe700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fde3402fd70 0x7fde34032230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.210+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde48059100 con 0x7fde48070430 2023-12-06T03:21:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.210+0000 7fde477fe700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fde3402fd70 0x7fde34032230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fde3c0096a0 tx=0x7fde3c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.215+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fde38016070 con 0x7fde48070430 2023-12-06T03:21:24.387 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.385+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fde480ffc00 con 0x7fde3402fd70 2023-12-06T03:21:24.433 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.432+0000 7fde44ff9700 1 -- 172.21.15.55:0/242198420 <== mgr.14164 v2:172.21.15.55:6800/785191101 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fde480ffc00 con 0x7fde3402fd70 2023-12-06T03:21:24.434 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: Self-signed certificate created 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fde3402fd70 msgr2=0x7fde34032230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fde3402fd70 0x7fde34032230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fde3c0096a0 tx=0x7fde3c006b40).stop 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 msgr2=0x7fde480fc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fc360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fde3800e040 tx=0x7fde38006ee0).stop 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 shutdown_connections 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fde3402fd70 0x7fde34032230 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 --2- 172.21.15.55:0/242198420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde48070430 0x7fde480fc360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 >> 172.21.15.55:0/242198420 conn(0x7fde480f63c0 msgr2=0x7fde480f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 shutdown_connections 2023-12-06T03:21:24.436 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:24.434+0000 7fde4e5cf700 1 -- 172.21.15.55:0/242198420 wait complete. 2023-12-06T03:21:24.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:23.406266+0000 mgr.smithi055.qjkkyb (mgr.14164) 1 : audit [DBG] from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-06T03:21:24.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:23.406901+0000 mon.smithi055 (mon.0) 103 : cluster [DBG] mgrmap e15: smithi055.qjkkyb(active, since 1.06108s) 2023-12-06T03:21:24.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:23.413769+0000 mgr.smithi055.qjkkyb (mgr.14164) 2 : audit [DBG] from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-06T03:21:24.795 INFO:teuthology.orchestra.run.smithi055.stderr:Creating initial admin user... 2023-12-06T03:21:24.796 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp_dnt77pd:/tmp/dashboard.pw:z docker.io/ceph/ceph:v16.2.4 dashboard ac-user-create admin -i /tmp/dashboard.pw administrator --force-password --pwd-update-required 2023-12-06T03:21:25.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.168+0000 7f588cc76700 1 Processor -- start 2023-12-06T03:21:25.170 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.168+0000 7f588cc76700 1 -- start start 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.168+0000 7f588cc76700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.168+0000 7f588cc76700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880fede0 con 0x7f58880704a0 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39086/0 (socket says 172.21.15.55:39086) 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588659c700 1 -- 172.21.15.55:0/3107402157 learned_addr learned my addr 172.21.15.55:0/3107402157 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588659c700 1 -- 172.21.15.55:0/3107402157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880fef20 con 0x7f58880704a0 2023-12-06T03:21:25.172 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588659c700 1 --2- 172.21.15.55:0/3107402157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fe8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f587c00b5f0 tx=0x7f587c006ee0).ready entity=mon.0 client_cookie=ee3d80a8d6a1725d server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588559a700 1 -- 172.21.15.55:0/3107402157 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c00d600 con 0x7f58880704a0 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588559a700 1 -- 172.21.15.55:0/3107402157 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f587c00dbd0 con 0x7f58880704a0 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.169+0000 7f588559a700 1 -- 172.21.15.55:0/3107402157 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c011c00 con 0x7f58880704a0 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 -- 172.21.15.55:0/3107402157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 msgr2=0x7f58880fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 --2- 172.21.15.55:0/3107402157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fe8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f587c00b5f0 tx=0x7f587c006ee0).stop 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 -- 172.21.15.55:0/3107402157 shutdown_connections 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 --2- 172.21.15.55:0/3107402157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fe8f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 -- 172.21.15.55:0/3107402157 >> 172.21.15.55:0/3107402157 conn(0x7f58880f63b0 msgr2=0x7f58880f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 -- 172.21.15.55:0/3107402157 shutdown_connections 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.170+0000 7f588cc76700 1 -- 172.21.15.55:0/3107402157 wait complete. 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588cc76700 1 Processor -- start 2023-12-06T03:21:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588cc76700 1 -- start start 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588cc76700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588cc76700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880fa7f0 con 0x7f58880704a0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39088/0 (socket says 172.21.15.55:39088) 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588659c700 1 -- 172.21.15.55:0/2018293334 learned_addr learned my addr 172.21.15.55:0/2018293334 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588659c700 1 -- 172.21.15.55:0/2018293334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c00b200 con 0x7f58880704a0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.171+0000 7f588659c700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fc370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f587c00e040 tx=0x7f587c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.172+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c00d9f0 con 0x7f58880704a0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.172+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f587c01a9f0 con 0x7f58880704a0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.172+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f587c01b8c0 con 0x7f58880704a0 2023-12-06T03:21:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.172+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58880fa990 con 0x7f58880704a0 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.172+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58880fada0 con 0x7f58880704a0 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.173+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f587c00f070 con 0x7f58880704a0 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.173+0000 7f5876ffd700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f586c02feb0 0x7f586c032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.173+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f587c041630 con 0x7f58880704a0 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.173+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5888045ad0 con 0x7f58880704a0 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.173+0000 7f5885d9b700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f586c02feb0 0x7f586c032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.174+0000 7f5885d9b700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f586c02feb0 0x7f586c032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f58780096a0 tx=0x7f5878006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.178+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f587c016070 con 0x7f58880704a0 2023-12-06T03:21:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.360+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5888058f20 con 0x7f586c02feb0 2023-12-06T03:21:25.616 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.614+0000 7f5876ffd700 1 -- 172.21.15.55:0/2018293334 <== mgr.14164 v2:172.21.15.55:6800/785191101 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f5888058f20 con 0x7f586c02feb0 2023-12-06T03:21:25.616 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$/F0T16/7jbqrBYMsafiQQOW38T5ywl5Q8bGvBt0W7PVNpAc31kylC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701832885, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-06T03:21:25.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f586c02feb0 msgr2=0x7f586c032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:25.618 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f586c02feb0 0x7f586c032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f58780096a0 tx=0x7f5878006b40).stop 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 msgr2=0x7f58880fc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fc370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f587c00e040 tx=0x7f587c006ee0).stop 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 shutdown_connections 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f586c02feb0 0x7f586c032370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 --2- 172.21.15.55:0/2018293334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58880704a0 0x7f58880fc370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 >> 172.21.15.55:0/2018293334 conn(0x7f58880f63b0 msgr2=0x7f58880f7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 shutdown_connections 2023-12-06T03:21:25.619 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:25.616+0000 7f588cc76700 1 -- 172.21.15.55:0/2018293334 wait complete. 2023-12-06T03:21:25.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:24.387020+0000 mgr.smithi055.qjkkyb (mgr.14164) 3 : audit [DBG] from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:25.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:24.430518+0000 mon.smithi055 (mon.0) 104 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:25.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:24.432909+0000 mon.smithi055 (mon.0) 105 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:25.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:24.971180+0000 mon.smithi055 (mon.0) 106 : cluster [DBG] mgrmap e16: smithi055.qjkkyb(active, since 2s) 2023-12-06T03:21:25.971 INFO:teuthology.orchestra.run.smithi055.stderr:Fetching dashboard port number... 2023-12-06T03:21:25.971 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmp_7luko1q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp5oc6c81u:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config get mgr mgr/dashboard/ssl_server_port 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.367+0000 7f78e86a5700 1 Processor -- start 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.368+0000 7f78e86a5700 1 -- start start 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.368+0000 7f78e86a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.368+0000 7f78e86a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00fee30 con 0x7f78e00703f0 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.368+0000 7f78e6441700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.368+0000 7f78e6441700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fe940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39098/0 (socket says 172.21.15.55:39098) 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.368+0000 7f78e6441700 1 -- 172.21.15.55:0/2293725922 learned_addr learned my addr 172.21.15.55:0/2293725922 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.369+0000 7f78e6441700 1 -- 172.21.15.55:0/2293725922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e00fef70 con 0x7f78e00703f0 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.369+0000 7f78e6441700 1 --2- 172.21.15.55:0/2293725922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fe940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f78dc00b5f0 tx=0x7f78dc006ee0).ready entity=mon.0 client_cookie=597fb522389c65e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.369+0000 7f78e543f700 1 -- 172.21.15.55:0/2293725922 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78dc00d600 con 0x7f78e00703f0 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.369+0000 7f78e543f700 1 -- 172.21.15.55:0/2293725922 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f78dc00dbd0 con 0x7f78e00703f0 2023-12-06T03:21:26.371 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.369+0000 7f78e543f700 1 -- 172.21.15.55:0/2293725922 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78dc011c00 con 0x7f78e00703f0 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 -- 172.21.15.55:0/2293725922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 msgr2=0x7f78e00fe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 --2- 172.21.15.55:0/2293725922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fe940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f78dc00b5f0 tx=0x7f78dc006ee0).stop 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 -- 172.21.15.55:0/2293725922 shutdown_connections 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 --2- 172.21.15.55:0/2293725922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fe940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 -- 172.21.15.55:0/2293725922 >> 172.21.15.55:0/2293725922 conn(0x7f78e00f63c0 msgr2=0x7f78e00f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 -- 172.21.15.55:0/2293725922 shutdown_connections 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.370+0000 7f78e86a5700 1 -- 172.21.15.55:0/2293725922 wait complete. 2023-12-06T03:21:26.373 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e86a5700 1 Processor -- start 2023-12-06T03:21:26.374 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e86a5700 1 -- start start 2023-12-06T03:21:26.374 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e86a5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:26.374 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e86a5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00fa840 con 0x7f78e00703f0 2023-12-06T03:21:26.374 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e6441700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:26.374 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e6441700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39110/0 (socket says 172.21.15.55:39110) 2023-12-06T03:21:26.374 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e6441700 1 -- 172.21.15.55:0/2261501481 learned_addr learned my addr 172.21.15.55:0/2261501481 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e6441700 1 -- 172.21.15.55:0/2261501481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc00b200 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.371+0000 7f78e6441700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fc3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f78dc00b190 tx=0x7f78dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.372+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78dc01b410 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.372+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f78dc01b9e0 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.372+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e00fa9e0 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.372+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f78dc01a880 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.372+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e00fadf0 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.373+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f78dc00f070 con 0x7f78e00703f0 2023-12-06T03:21:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.373+0000 7f78ceffd700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f78c802feb0 0x7f78c8032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:26.376 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.373+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f78dc0414b0 con 0x7f78e00703f0 2023-12-06T03:21:26.376 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.373+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e0045ad0 con 0x7f78e00703f0 2023-12-06T03:21:26.376 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.373+0000 7f78e5c40700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f78c802feb0 0x7f78c8032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:26.376 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.373+0000 7f78e5c40700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f78c802feb0 0x7f78c8032370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f78d00096a0 tx=0x7f78d0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.377+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f78dc016070 con 0x7f78e00703f0 2023-12-06T03:21:26.542 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.540+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f78e00ffbf0 con 0x7f78e00703f0 2023-12-06T03:21:26.542 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.541+0000 7f78ceffd700 1 -- 172.21.15.55:0/2261501481 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v9) v1 ==== 112+0+5 (secure 0 0 0) 0x7f78dc019e60 con 0x7f78e00703f0 2023-12-06T03:21:26.542 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 8443 2023-12-06T03:21:26.544 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f78c802feb0 msgr2=0x7f78c8032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:26.544 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f78c802feb0 0x7f78c8032370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f78d00096a0 tx=0x7f78d0006b40).stop 2023-12-06T03:21:26.544 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 msgr2=0x7f78e00fc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:26.544 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fc3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f78dc00b190 tx=0x7f78dc006ee0).stop 2023-12-06T03:21:26.544 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 shutdown_connections 2023-12-06T03:21:26.545 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f78c802feb0 0x7f78c8032370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:26.545 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 --2- 172.21.15.55:0/2261501481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78e00703f0 0x7f78e00fc3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:26.545 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.542+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 >> 172.21.15.55:0/2261501481 conn(0x7f78e00f63c0 msgr2=0x7f78e00f7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:26.545 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.543+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 shutdown_connections 2023-12-06T03:21:26.545 INFO:teuthology.orchestra.run.smithi055.stderr:/usr/bin/ceph: 2023-12-06T03:21:26.543+0000 7f78e86a5700 1 -- 172.21.15.55:0/2261501481 wait complete. 2023-12-06T03:21:26.839 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:25.362021+0000 mgr.smithi055.qjkkyb (mgr.14164) 4 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:26.840 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:25.615668+0000 mon.smithi055 (mon.0) 107 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:26.840 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:26.541997+0000 mon.smithi055 (mon.0) 108 : audit [DBG] from='client.? 172.21.15.55:0/2261501481' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-06T03:21:26.912 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-enabled firewalld.service 2023-12-06T03:21:26.923 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: disabled 2023-12-06T03:21:26.924 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: systemctl is-active firewalld.service 2023-12-06T03:21:26.932 INFO:teuthology.orchestra.run.smithi055.stderr:systemctl: inactive 2023-12-06T03:21:26.933 INFO:teuthology.orchestra.run.smithi055.stderr:firewalld.service is not enabled 2023-12-06T03:21:26.933 INFO:teuthology.orchestra.run.smithi055.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-06T03:21:26.936 INFO:teuthology.orchestra.run.smithi055.stderr:Ceph Dashboard is now available at: 2023-12-06T03:21:26.936 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.936 INFO:teuthology.orchestra.run.smithi055.stderr: URL: https://smithi055.front.sepia.ceph.com:8443/ 2023-12-06T03:21:26.936 INFO:teuthology.orchestra.run.smithi055.stderr: User: admin 2023-12-06T03:21:26.936 INFO:teuthology.orchestra.run.smithi055.stderr: Password: xtgoray6qx 2023-12-06T03:21:26.936 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr:You can access the Ceph CLI with: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr:Please consider enabling telemetry to help improve Ceph: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: ceph telemetry on 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr:For more information see: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.937 INFO:teuthology.orchestra.run.smithi055.stderr: https://docs.ceph.com/docs/pacific/mgr/telemetry/ 2023-12-06T03:21:26.938 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:21:26.938 INFO:teuthology.orchestra.run.smithi055.stderr:Bootstrap complete. 2023-12-06T03:21:26.938 INFO:teuthology.orchestra.run.smithi055.stderr:Releasing lock 140226314443240 on /run/cephadm/5555246c-93e6-11ee-95a2-87774f69a715.lock 2023-12-06T03:21:26.938 INFO:teuthology.orchestra.run.smithi055.stderr:Lock 140226314443240 released on /run/cephadm/5555246c-93e6-11ee-95a2-87774f69a715.lock 2023-12-06T03:21:26.978 INFO:tasks.cephadm:Fetching config... 2023-12-06T03:21:26.978 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:21:26.978 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-06T03:21:27.041 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-06T03:21:27.041 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:21:27.041 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-06T03:21:27.104 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-06T03:21:27.104 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:21:27.104 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/mon.smithi055/keyring of=/dev/stdout 2023-12-06T03:21:27.179 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-06T03:21:27.179 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-12-06T03:21:27.179 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-06T03:21:27.242 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-06T03:21:27.242 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC59hmyDdUdG6zA//0cDdjwFDZenhT3qyJ0LflT/m9Ww296uBah/pXwb0zmfwMJBZ/aFdTWI839qUokYof+LYFdOfVvR6aBc1AzYDFWj0UgOEM14xMDieAvNIbVhRqFaJz/DJO0LASJV1HscS0gWUBIka0zAgqLNueNZH0l7rcf6eVGprjz8iFTfn2OaXswQu92Xhi87HYFKyzRYIZr/SsbJSOBNggo65iq52pPgbJx0iex7qobYCyN0MngFYgPTMBZr/6ZnjH2+CYnPDsn3DbFTcE6eUAX/tWBcEm+FQrZZm1Ohbg9FOHD8iizhKiGji3UGo3pCHMw2OgBEKAzxSg3IspJ3iVnfj+yUqt0QotsdQ/GAh8YntjgGQoqjUzcCcrNjM8CpzOOkt/N3VMYkxe9YQgUw2cDR5G+xpeNJf8fjF5oiTVYj3wtqSJtf4BLWznjharzUjaXGIXdTzYQ6HB3PjC15XmF/KVBG8gveuN2U7r5nZKc1RRSZnw2foY7/30= ceph-5555246c-93e6-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-06T03:21:27.328 INFO:teuthology.orchestra.run.smithi055.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC59hmyDdUdG6zA//0cDdjwFDZenhT3qyJ0LflT/m9Ww296uBah/pXwb0zmfwMJBZ/aFdTWI839qUokYof+LYFdOfVvR6aBc1AzYDFWj0UgOEM14xMDieAvNIbVhRqFaJz/DJO0LASJV1HscS0gWUBIka0zAgqLNueNZH0l7rcf6eVGprjz8iFTfn2OaXswQu92Xhi87HYFKyzRYIZr/SsbJSOBNggo65iq52pPgbJx0iex7qobYCyN0MngFYgPTMBZr/6ZnjH2+CYnPDsn3DbFTcE6eUAX/tWBcEm+FQrZZm1Ohbg9FOHD8iizhKiGji3UGo3pCHMw2OgBEKAzxSg3IspJ3iVnfj+yUqt0QotsdQ/GAh8YntjgGQoqjUzcCcrNjM8CpzOOkt/N3VMYkxe9YQgUw2cDR5G+xpeNJf8fjF5oiTVYj3wtqSJtf4BLWznjharzUjaXGIXdTzYQ6HB3PjC15XmF/KVBG8gveuN2U7r5nZKc1RRSZnw2foY7/30= ceph-5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:21:27.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC59hmyDdUdG6zA//0cDdjwFDZenhT3qyJ0LflT/m9Ww296uBah/pXwb0zmfwMJBZ/aFdTWI839qUokYof+LYFdOfVvR6aBc1AzYDFWj0UgOEM14xMDieAvNIbVhRqFaJz/DJO0LASJV1HscS0gWUBIka0zAgqLNueNZH0l7rcf6eVGprjz8iFTfn2OaXswQu92Xhi87HYFKyzRYIZr/SsbJSOBNggo65iq52pPgbJx0iex7qobYCyN0MngFYgPTMBZr/6ZnjH2+CYnPDsn3DbFTcE6eUAX/tWBcEm+FQrZZm1Ohbg9FOHD8iizhKiGji3UGo3pCHMw2OgBEKAzxSg3IspJ3iVnfj+yUqt0QotsdQ/GAh8YntjgGQoqjUzcCcrNjM8CpzOOkt/N3VMYkxe9YQgUw2cDR5G+xpeNJf8fjF5oiTVYj3wtqSJtf4BLWznjharzUjaXGIXdTzYQ6HB3PjC15XmF/KVBG8gveuN2U7r5nZKc1RRSZnw2foY7/30= ceph-5555246c-93e6-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-06T03:21:27.389 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC59hmyDdUdG6zA//0cDdjwFDZenhT3qyJ0LflT/m9Ww296uBah/pXwb0zmfwMJBZ/aFdTWI839qUokYof+LYFdOfVvR6aBc1AzYDFWj0UgOEM14xMDieAvNIbVhRqFaJz/DJO0LASJV1HscS0gWUBIka0zAgqLNueNZH0l7rcf6eVGprjz8iFTfn2OaXswQu92Xhi87HYFKyzRYIZr/SsbJSOBNggo65iq52pPgbJx0iex7qobYCyN0MngFYgPTMBZr/6ZnjH2+CYnPDsn3DbFTcE6eUAX/tWBcEm+FQrZZm1Ohbg9FOHD8iizhKiGji3UGo3pCHMw2OgBEKAzxSg3IspJ3iVnfj+yUqt0QotsdQ/GAh8YntjgGQoqjUzcCcrNjM8CpzOOkt/N3VMYkxe9YQgUw2cDR5G+xpeNJf8fjF5oiTVYj3wtqSJtf4BLWznjharzUjaXGIXdTzYQ6HB3PjC15XmF/KVBG8gveuN2U7r5nZKc1RRSZnw2foY7/30= ceph-5555246c-93e6-11ee-95a2-87774f69a715 2023-12-06T03:21:27.401 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-06T03:21:28.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/279807440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 msgr2=0x7fbdc40f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:28.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 --2- 172.21.15.55:0/279807440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc40f6930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fbdb4009fb0 tx=0x7fbdb4009b30).stop 2023-12-06T03:21:28.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/279807440 shutdown_connections 2023-12-06T03:21:28.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 --2- 172.21.15.55:0/279807440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc40f6930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:28.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/279807440 >> 172.21.15.55:0/279807440 conn(0x7fbdc40f1c70 msgr2=0x7fbdc40f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:28.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/279807440 shutdown_connections 2023-12-06T03:21:28.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.598+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/279807440 wait complete. 2023-12-06T03:21:28.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.599+0000 7fbdc9d6b700 1 Processor -- start 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.599+0000 7fbdc9d6b700 1 -- start start 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.599+0000 7fbdc9d6b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc41088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.599+0000 7fbdc9d6b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc4108de0 con 0x7fbdc40f6510 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc41088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc41088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39138/0 (socket says 172.21.15.55:39138) 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc37fe700 1 -- 172.21.15.55:0/2738670697 learned_addr learned my addr 172.21.15.55:0/2738670697 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:28.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc37fe700 1 -- 172.21.15.55:0/2738670697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4005040 con 0x7fbdc40f6510 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc37fe700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc41088f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fbdb4004dd0 tx=0x7fbdb400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdb4009320 con 0x7fbdc40f6510 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fbdb4013680 con 0x7fbdc40f6510 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc4108fe0 con 0x7fbdc40f6510 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdb401ecb0 con 0x7fbdc40f6510 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.600+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc4109470 con 0x7fbdc40f6510 2023-12-06T03:21:28.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.601+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fbdb4014420 con 0x7fbdc40f6510 2023-12-06T03:21:28.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.601+0000 7fbdabfff700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fbdac02ff60 0x7fbdac032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:28.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.601+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fbdb4041100 con 0x7fbdc40f6510 2023-12-06T03:21:28.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.601+0000 7fbdc2ffd700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fbdac02ff60 0x7fbdac032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:28.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.601+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc4059100 con 0x7fbdc40f6510 2023-12-06T03:21:28.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.602+0000 7fbdc2ffd700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fbdac02ff60 0x7fbdac032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbdb80096a0 tx=0x7fbdb8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:28.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.606+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbdb4027490 con 0x7fbdc40f6510 2023-12-06T03:21:28.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.772+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fbdc4045ad0 con 0x7fbdc40f6510 2023-12-06T03:21:28.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.775+0000 7fbdabfff700 1 -- 172.21.15.55:0/2738670697 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v10)=0 v10) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbdb401c6e0 con 0x7fbdc40f6510 2023-12-06T03:21:28.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fbdac02ff60 msgr2=0x7fbdac032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:28.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fbdac02ff60 0x7fbdac032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbdb80096a0 tx=0x7fbdb8006b40).stop 2023-12-06T03:21:28.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 msgr2=0x7fbdc41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:28.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc41088f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fbdb4004dd0 tx=0x7fbdb400b040).stop 2023-12-06T03:21:28.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 shutdown_connections 2023-12-06T03:21:28.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fbdac02ff60 0x7fbdac032420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:28.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 --2- 172.21.15.55:0/2738670697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbdc40f6510 0x7fbdc41088f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:28.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.777+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 >> 172.21.15.55:0/2738670697 conn(0x7fbdc40f1c70 msgr2=0x7fbdc40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:28.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.778+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 shutdown_connections 2023-12-06T03:21:28.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:28.778+0000 7fbdc9d6b700 1 -- 172.21.15.55:0/2738670697 wait complete. 2023-12-06T03:21:29.207 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2023-12-06T03:21:29.207 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:21:29.207 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2023-12-06T03:21:29.229 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2023-12-06T03:21:29.230 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-06T03:21:29.291 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2023-12-06T03:21:29.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph orch host add smithi175 2023-12-06T03:21:29.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:28.437661+0000 mon.smithi055 (mon.0) 109 : cluster [DBG] mgrmap e17: smithi055.qjkkyb(active, since 6s) 2023-12-06T03:21:29.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:28.776646+0000 mon.smithi055 (mon.0) 110 : audit [INF] from='client.? 172.21.15.55:0/2738670697' entity='client.admin' 2023-12-06T03:21:31.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:30.579430+0000 mon.smithi055 (mon.0) 111 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:31.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:30.827643+0000 mon.smithi055 (mon.0) 112 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:31.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:31.508754+0000 mon.smithi055 (mon.0) 113 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:34 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:33.985833+0000 mon.smithi055 (mon.0) 114 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:34 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:34.280877+0000 mon.smithi055 (mon.0) 115 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:34 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:34.285774+0000 mon.smithi055 (mon.0) 116 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:34 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:34.292543+0000 mgr.smithi055.qjkkyb (mgr.14164) 5 : cephadm [INF] Deploying daemon alertmanager.smithi055 on smithi055 2023-12-06T03:21:43.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:42.404882+0000 mgr.smithi055.qjkkyb (mgr.14164) 6 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:44 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:44.405261+0000 mgr.smithi055.qjkkyb (mgr.14164) 7 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:46.405610+0000 mgr.smithi055.qjkkyb (mgr.14164) 8 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:48.405896+0000 mgr.smithi055.qjkkyb (mgr.14164) 9 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:50.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 -- 172.21.15.175:0/1916306006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 msgr2=0x7fc7080f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 --2- 172.21.15.175:0/1916306006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc7080f5830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fc6fc009fb0 tx=0x7fc6fc009b30).stop 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 -- 172.21.15.175:0/1916306006 shutdown_connections 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 --2- 172.21.15.175:0/1916306006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc7080f5830 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 -- 172.21.15.175:0/1916306006 >> 172.21.15.175:0/1916306006 conn(0x7fc7080f0b70 msgr2=0x7fc7080f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 -- 172.21.15.175:0/1916306006 shutdown_connections 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.082+0000 7fc70ce57700 1 -- 172.21.15.175:0/1916306006 wait complete. 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70ce57700 1 Processor -- start 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70ce57700 1 -- start start 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70ce57700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc708109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70ce57700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc708060220 con 0x7fc7080f5410 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc708109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc708109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56922/0 (socket says 172.21.15.175:56922) 2023-12-06T03:21:50.085 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.083+0000 7fc70659c700 1 -- 172.21.15.175:0/3814465109 learned_addr learned my addr 172.21.15.175:0/3814465109 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:21:50.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc70659c700 1 -- 172.21.15.175:0/3814465109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc005040 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc70659c700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc708109450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fc6fc013510 tx=0x7fc6fc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6fc009320 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6fc013f00 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708060420 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6fc014d90 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.084+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708109b00 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.085+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fc6fc0095f0 con 0x7fc7080f5410 2023-12-06T03:21:50.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.085+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708059100 con 0x7fc7080f5410 2023-12-06T03:21:50.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.085+0000 7fc6f6ffd700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc6f002ffb0 0x7fc6f0032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:50.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.085+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fc6fc040f80 con 0x7fc7080f5410 2023-12-06T03:21:50.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.085+0000 7fc705d9b700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc6f002ffb0 0x7fc6f0032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:50.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.086+0000 7fc705d9b700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc6f002ffb0 0x7fc6f0032470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fc6f80096a0 tx=0x7fc6f8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:50.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.090+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc6fc014420 con 0x7fc7080f5410 2023-12-06T03:21:50.261 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.260+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7080f1d70 con 0x7fc6f002ffb0 2023-12-06T03:21:50.887 INFO:teuthology.orchestra.run.smithi175.stdout:Added host 'smithi175' 2023-12-06T03:21:50.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.883+0000 7fc6f6ffd700 1 -- 172.21.15.175:0/3814465109 <== mgr.14164 v2:172.21.15.55:6800/785191101 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7fc7080f1d70 con 0x7fc6f002ffb0 2023-12-06T03:21:50.888 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc6f002ffb0 msgr2=0x7fc6f0032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc6f002ffb0 0x7fc6f0032470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fc6f80096a0 tx=0x7fc6f8006b40).stop 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 msgr2=0x7fc708109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc708109450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fc6fc013510 tx=0x7fc6fc00b040).stop 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 shutdown_connections 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc6f002ffb0 0x7fc6f0032470 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 --2- 172.21.15.175:0/3814465109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7080f5410 0x7fc708109450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 >> 172.21.15.175:0/3814465109 conn(0x7fc7080f0b70 msgr2=0x7fc7080f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 shutdown_connections 2023-12-06T03:21:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:50.886+0000 7fc70ce57700 1 -- 172.21.15.175:0/3814465109 wait complete. 2023-12-06T03:21:51.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-12-06T03:21:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:50.262569+0000 mgr.smithi055.qjkkyb (mgr.14164) 10 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:51.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:50.406252+0000 mgr.smithi055.qjkkyb (mgr.14164) 11 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:51.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:50.883777+0000 mon.smithi055 (mon.0) 117 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:52.370 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 -- 172.21.15.175:0/100887651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 msgr2=0x7fce780f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:52.370 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 --2- 172.21.15.175:0/100887651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce780f90f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fce60009fb0 tx=0x7fce60009b30).stop 2023-12-06T03:21:52.371 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 -- 172.21.15.175:0/100887651 shutdown_connections 2023-12-06T03:21:52.371 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 --2- 172.21.15.175:0/100887651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce780f90f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:52.371 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 -- 172.21.15.175:0/100887651 >> 172.21.15.175:0/100887651 conn(0x7fce780f0b70 msgr2=0x7fce780f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:52.371 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 -- 172.21.15.175:0/100887651 shutdown_connections 2023-12-06T03:21:52.371 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.369+0000 7fce7deec700 1 -- 172.21.15.175:0/100887651 wait complete. 2023-12-06T03:21:52.372 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce7deec700 1 Processor -- start 2023-12-06T03:21:52.372 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce7deec700 1 -- start start 2023-12-06T03:21:52.372 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce7deec700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce781066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:52.372 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce7deec700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78106bd0 con 0x7fce78060280 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce781066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce781066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56938/0 (socket says 172.21.15.175:56938) 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.370+0000 7fce777fe700 1 -- 172.21.15.175:0/3185366469 learned_addr learned my addr 172.21.15.175:0/3185366469 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce777fe700 1 -- 172.21.15.175:0/3185366469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60005040 con 0x7fce78060280 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce777fe700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce781066e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fce60013510 tx=0x7fce6000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce60009320 con 0x7fce78060280 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78106dd0 con 0x7fce78060280 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fce60013f00 con 0x7fce78060280 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce60014dc0 con 0x7fce78060280 2023-12-06T03:21:52.373 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.371+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce78107260 con 0x7fce78060280 2023-12-06T03:21:52.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.372+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fce60014420 con 0x7fce78060280 2023-12-06T03:21:52.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.372+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce78045ad0 con 0x7fce78060280 2023-12-06T03:21:52.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.372+0000 7fce6ffff700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fce6402ff60 0x7fce64032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:52.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.372+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fce60041050 con 0x7fce78060280 2023-12-06T03:21:52.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.372+0000 7fce76ffd700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fce6402ff60 0x7fce64032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:52.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.373+0000 7fce76ffd700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fce6402ff60 0x7fce64032420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fce680096a0 tx=0x7fce68006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:52.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.377+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fce600095f0 con 0x7fce78060280 2023-12-06T03:21:52.549 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.548+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce780f1d70 con 0x7fce6402ff60 2023-12-06T03:21:52.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.549+0000 7fce6ffff700 1 -- 172.21.15.175:0/3185366469 <== mgr.14164 v2:172.21.15.55:6800/785191101 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+156 (secure 0 0 0) 0x7fce780f1d70 con 0x7fce6402ff60 2023-12-06T03:21:52.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:21:52.551 INFO:teuthology.orchestra.run.smithi175.stdout:[{"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "smithi175", "hostname": "smithi175", "labels": [], "status": ""}] 2023-12-06T03:21:52.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fce6402ff60 msgr2=0x7fce64032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:52.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fce6402ff60 0x7fce64032420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fce680096a0 tx=0x7fce68006b40).stop 2023-12-06T03:21:52.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 msgr2=0x7fce781066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:52.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce781066e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fce60013510 tx=0x7fce6000b040).stop 2023-12-06T03:21:52.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 shutdown_connections 2023-12-06T03:21:52.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fce6402ff60 0x7fce64032420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:52.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 --2- 172.21.15.175:0/3185366469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce78060280 0x7fce781066e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:52.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 >> 172.21.15.175:0/3185366469 conn(0x7fce780f0b70 msgr2=0x7fce780f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:52.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 shutdown_connections 2023-12-06T03:21:52.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:52.551+0000 7fce7deec700 1 -- 172.21.15.175:0/3185366469 wait complete. 2023-12-06T03:21:52.924 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-06T03:21:52.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-12-06T03:21:52.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:50.884216+0000 mgr.smithi055.qjkkyb (mgr.14164) 12 : cephadm [INF] Added host smithi175 2023-12-06T03:21:53.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:52.406510+0000 mgr.smithi055.qjkkyb (mgr.14164) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:53.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:52.549981+0000 mgr.smithi055.qjkkyb (mgr.14164) 14 : audit [DBG] from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-06T03:21:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 -- 172.21.15.55:0/4220959623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 msgr2=0x7f7a280f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 --2- 172.21.15.55:0/4220959623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a280f55c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7a1c009fb0 tx=0x7f7a1c009b30).stop 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 -- 172.21.15.55:0/4220959623 shutdown_connections 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 --2- 172.21.15.55:0/4220959623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a280f55c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 -- 172.21.15.55:0/4220959623 >> 172.21.15.55:0/4220959623 conn(0x7f7a280f09a0 msgr2=0x7f7a280f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 -- 172.21.15.55:0/4220959623 shutdown_connections 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.268+0000 7f7a2dfae700 1 -- 172.21.15.55:0/4220959623 wait complete. 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2dfae700 1 Processor -- start 2023-12-06T03:21:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2dfae700 1 -- start start 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2dfae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a281064d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2dfae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a281069c0 con 0x7f7a280f51a0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2cfac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a281064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2cfac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a281064d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49868/0 (socket says 172.21.15.55:49868) 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2cfac700 1 -- 172.21.15.55:0/1379382484 learned_addr learned my addr 172.21.15.55:0/1379382484 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.269+0000 7f7a2cfac700 1 -- 172.21.15.55:0/1379382484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c005040 con 0x7f7a280f51a0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.270+0000 7f7a2cfac700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a281064d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f7a1c013510 tx=0x7f7a1c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.270+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a1c009320 con 0x7f7a280f51a0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.270+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a1c013f00 con 0x7f7a280f51a0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.270+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a28106bc0 con 0x7f7a280f51a0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.270+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a1c014dc0 con 0x7f7a280f51a0 2023-12-06T03:21:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.270+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a28107050 con 0x7f7a280f51a0 2023-12-06T03:21:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.271+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7a1c0263a0 con 0x7f7a280f51a0 2023-12-06T03:21:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.271+0000 7f7a257fa700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1802ffb0 0x7f7a18032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.271+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7a1c040ff0 con 0x7f7a280f51a0 2023-12-06T03:21:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.271+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a28045ad0 con 0x7f7a280f51a0 2023-12-06T03:21:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.271+0000 7f7a27fff700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1802ffb0 0x7f7a18032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.271+0000 7f7a27fff700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1802ffb0 0x7f7a18032470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7a140096a0 tx=0x7f7a14006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:54.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.276+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7a1c012920 con 0x7f7a280f51a0 2023-12-06T03:21:54.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.463+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7a28107d70 con 0x7f7a280f51a0 2023-12-06T03:21:54.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.978+0000 7f7a257fa700 1 -- 172.21.15.55:0/1379382484 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v5) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7a1c02b030 con 0x7f7a280f51a0 2023-12-06T03:21:54.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1802ffb0 msgr2=0x7f7a18032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:54.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1802ffb0 0x7f7a18032470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7a140096a0 tx=0x7f7a14006b40).stop 2023-12-06T03:21:54.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 msgr2=0x7f7a281064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a281064d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f7a1c013510 tx=0x7f7a1c00b040).stop 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 shutdown_connections 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1802ffb0 0x7f7a18032470 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 --2- 172.21.15.55:0/1379382484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a280f51a0 0x7f7a281064d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 >> 172.21.15.55:0/1379382484 conn(0x7f7a280f09a0 msgr2=0x7f7a2805fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 shutdown_connections 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-12-06T03:21:54.981+0000 7f7a2dfae700 1 -- 172.21.15.55:0/1379382484 wait complete. 2023-12-06T03:21:54.982 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2023-12-06T03:21:55.298 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:54 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:54.406756+0000 mgr.smithi055.qjkkyb (mgr.14164) 15 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:55.299 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:54 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:54.463378+0000 mon.smithi055 (mon.0) 118 : audit [INF] from='client.? 172.21.15.55:0/1379382484' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-06T03:21:55.452 INFO:tasks.cephadm:Adding mon.smithi055 on smithi055 2023-12-06T03:21:55.452 INFO:tasks.cephadm:Adding mon.smithi175 on smithi175 2023-12-06T03:21:55.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph orch apply mon '2;smithi055:172.21.15.55=smithi055;smithi175:172.21.15.175=smithi175' 2023-12-06T03:21:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:54.978905+0000 mon.smithi055 (mon.0) 119 : audit [INF] from='client.? 172.21.15.55:0/1379382484' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-06T03:21:56.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:54.978977+0000 mon.smithi055 (mon.0) 120 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2023-12-06T03:21:56.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- 172.21.15.175:0/1277276958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 msgr2=0x7f9ea40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 --2- 172.21.15.175:0/1277276958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea40f5830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f9e98009fb0 tx=0x7f9e98009b30).stop 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- 172.21.15.175:0/1277276958 shutdown_connections 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 --2- 172.21.15.175:0/1277276958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea40f5830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- 172.21.15.175:0/1277276958 >> 172.21.15.175:0/1277276958 conn(0x7f9ea40f0b70 msgr2=0x7f9ea40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- 172.21.15.175:0/1277276958 shutdown_connections 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- 172.21.15.175:0/1277276958 wait complete. 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 Processor -- start 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- start start 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea4106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:56.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.526+0000 7f9eaae52700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4106c20 con 0x7f9ea40f5410 2023-12-06T03:21:56.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea8bee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea4106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:56.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea8bee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea4106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39522/0 (socket says 172.21.15.175:39522) 2023-12-06T03:21:56.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea8bee700 1 -- 172.21.15.175:0/3904705421 learned_addr learned my addr 172.21.15.175:0/3904705421 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:21:56.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea8bee700 1 -- 172.21.15.175:0/3904705421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98005040 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea8bee700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea4106730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9e98013510 tx=0x7f9e9800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e98009320 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea4106e20 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e98013f00 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea41072b0 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.527+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e98014db0 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.528+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9e980095f0 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.528+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea4045ad0 con 0x7f9ea40f5410 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.528+0000 7f9ea17fa700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9e9402ff60 0x7f9e94032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:56.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.528+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9e98041040 con 0x7f9ea40f5410 2023-12-06T03:21:56.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.529+0000 7f9ea3fff700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9e9402ff60 0x7f9e94032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:56.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.529+0000 7f9ea3fff700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9e9402ff60 0x7f9e94032420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9e900096a0 tx=0x7f9e90006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:56.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.533+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9e98014420 con 0x7f9ea40f5410 2023-12-06T03:21:56.712 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.710+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 --> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi055:172.21.15.55=smithi055;smithi175:172.21.15.175=smithi175", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ea40f1e20 con 0x7f9e9402ff60 2023-12-06T03:21:56.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.717+0000 7f9ea17fa700 1 -- 172.21.15.175:0/3904705421 <== mgr.14164 v2:172.21.15.55:6800/785191101 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9ea40f1e20 con 0x7f9e9402ff60 2023-12-06T03:21:56.718 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2023-12-06T03:21:56.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9e9402ff60 msgr2=0x7f9e94032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9e9402ff60 0x7f9e94032420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9e900096a0 tx=0x7f9e90006b40).stop 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 msgr2=0x7f9ea4106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea4106730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f9e98013510 tx=0x7f9e9800b040).stop 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 shutdown_connections 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9e9402ff60 0x7f9e94032420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 --2- 172.21.15.175:0/3904705421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ea40f5410 0x7f9ea4106730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 >> 172.21.15.175:0/3904705421 conn(0x7f9ea40f0b70 msgr2=0x7f9ea40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 shutdown_connections 2023-12-06T03:21:56.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:56.718+0000 7f9eaae52700 1 -- 172.21.15.175:0/3904705421 wait complete. 2023-12-06T03:21:57.102 DEBUG:teuthology.orchestra.run.smithi175:mon.smithi175> sudo journalctl -f -n 0 -u ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175.service 2023-12-06T03:21:57.104 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:21:57.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:21:57.133 INFO:journalctl@ceph.mon.smithi175.smithi175.stdout:-- Logs begin at Wed 2023-12-06 03:07:26 UTC. -- 2023-12-06T03:21:57.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:56.407109+0000 mgr.smithi055.qjkkyb (mgr.14164) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:57.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:56.717079+0000 mon.smithi055 (mon.0) 121 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:21:58.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 -- 172.21.15.175:0/1885063636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46680f5410 msgr2=0x7f46680f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 --2- 172.21.15.175:0/1885063636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46680f5410 0x7f46680f5830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f465c009fb0 tx=0x7f465c009b30).stop 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 -- 172.21.15.175:0/1885063636 shutdown_connections 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 --2- 172.21.15.175:0/1885063636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46680f5410 0x7f46680f5830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 -- 172.21.15.175:0/1885063636 >> 172.21.15.175:0/1885063636 conn(0x7f46680f0b70 msgr2=0x7f46680f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 -- 172.21.15.175:0/1885063636 shutdown_connections 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.282+0000 7f466db12700 1 -- 172.21.15.175:0/1885063636 wait complete. 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.283+0000 7f466db12700 1 Processor -- start 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.283+0000 7f466db12700 1 -- start start 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.283+0000 7f466db12700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 0x7f4668105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.283+0000 7f466db12700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668106050 con 0x7f4668105740 2023-12-06T03:21:58.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f46677fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 0x7f4668105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f46677fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 0x7f4668105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39538/0 (socket says 172.21.15.175:39538) 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f46677fe700 1 -- 172.21.15.175:0/1655935104 learned_addr learned my addr 172.21.15.175:0/1655935104 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f46677fe700 1 -- 172.21.15.175:0/1655935104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c005040 con 0x7f4668105740 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f46677fe700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 0x7f4668105b60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f465c009f80 tx=0x7f465c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f465c0136c0 con 0x7f4668105740 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4668106250 con 0x7f4668105740 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f465c013e70 con 0x7f4668105740 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f465c014420 con 0x7f4668105740 2023-12-06T03:21:58.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.284+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4668106da0 con 0x7f4668105740 2023-12-06T03:21:58.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.285+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f465c025060 con 0x7f4668105740 2023-12-06T03:21:58.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.285+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668059100 con 0x7f4668105740 2023-12-06T03:21:58.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.285+0000 7f464ffff700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f465002ff60 0x7f4650032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:21:58.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.285+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f465c040f00 con 0x7f4668105740 2023-12-06T03:21:58.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.285+0000 7f4666ffd700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f465002ff60 0x7f4650032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:21:58.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.286+0000 7f4666ffd700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f465002ff60 0x7f4650032420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f46580096a0 tx=0x7f4658006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:21:58.291 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.290+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f465c01cd10 con 0x7f4668105740 2023-12-06T03:21:58.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:56.712670+0000 mgr.smithi055.qjkkyb (mgr.14164) 17 : audit [DBG] from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi055:172.21.15.55=smithi055;smithi175:172.21.15.175=smithi175", "target": ["mon-mgr", ""]}]: dispatch 2023-12-06T03:21:58.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:21:56.714188+0000 mgr.smithi055.qjkkyb (mgr.14164) 18 : cephadm [INF] Saving service mon spec with placement smithi055:172.21.15.55=smithi055;smithi175:172.21.15.175=smithi175;count:2 2023-12-06T03:21:58.500 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.499+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4668045ad0 con 0x7f4668105740 2023-12-06T03:21:58.501 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.500+0000 7f464ffff700 1 -- 172.21.15.175:0/1655935104 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f465c01cb30 con 0x7f4668105740 2023-12-06T03:21:58.501 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:21:58.501 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f465002ff60 msgr2=0x7f4650032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f465002ff60 0x7f4650032420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f46580096a0 tx=0x7f4658006b40).stop 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 msgr2=0x7f4668105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 0x7f4668105b60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f465c009f80 tx=0x7f465c009b30).stop 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 shutdown_connections 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f465002ff60 0x7f4650032420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 --2- 172.21.15.175:0/1655935104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4668105740 0x7f4668105b60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 >> 172.21.15.175:0/1655935104 conn(0x7f46680f0b70 msgr2=0x7f46680f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 shutdown_connections 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:21:58.501+0000 7f466db12700 1 -- 172.21.15.175:0/1655935104 wait complete. 2023-12-06T03:21:58.503 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:21:59.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:58 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:21:58.407418+0000 mgr.smithi055.qjkkyb (mgr.14164) 19 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:21:59.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:21:58 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:21:58.500917+0000 mon.smithi055 (mon.0) 122 : audit [DBG] from='client.? 172.21.15.175:0/1655935104' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:21:59.876 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:21:59.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:00.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 -- 172.21.15.175:0/1048839820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 msgr2=0x7f2d980f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:00.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 --2- 172.21.15.175:0/1048839820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d980f5830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2d88009fb0 tx=0x7f2d88009b30).stop 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 -- 172.21.15.175:0/1048839820 shutdown_connections 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 --2- 172.21.15.175:0/1048839820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d980f5830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 -- 172.21.15.175:0/1048839820 >> 172.21.15.175:0/1048839820 conn(0x7f2d980f0b70 msgr2=0x7f2d980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 -- 172.21.15.175:0/1048839820 shutdown_connections 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.964+0000 7f2d9ea43700 1 -- 172.21.15.175:0/1048839820 wait complete. 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.965+0000 7f2d9ea43700 1 Processor -- start 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.965+0000 7f2d9ea43700 1 -- start start 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.965+0000 7f2d9ea43700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d981093d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:00.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.965+0000 7f2d9ea43700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98060240 con 0x7f2d980f5410 2023-12-06T03:22:00.968 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d97fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d981093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d97fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d981093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39548/0 (socket says 172.21.15.175:39548) 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d97fff700 1 -- 172.21.15.175:0/3984150583 learned_addr learned my addr 172.21.15.175:0/3984150583 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d97fff700 1 -- 172.21.15.175:0/3984150583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d88005040 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d97fff700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d981093d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2d88014000 tx=0x7f2d8800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d880095f0 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d98060440 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d88007e70 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d88014ea0 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.966+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d98109a80 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.967+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2d88025070 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.967+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d98045ad0 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.967+0000 7f2d94ff9700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2d8402ff60 0x7f2d84032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.967+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2d88040a30 con 0x7f2d980f5410 2023-12-06T03:22:00.969 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.968+0000 7f2d977fe700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2d8402ff60 0x7f2d84032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:00.970 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.968+0000 7f2d977fe700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2d8402ff60 0x7f2d84032420 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f2d800096a0 tx=0x7f2d80006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:00.972+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2d880079c0 con 0x7f2d980f5410 2023-12-06T03:22:01.182 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.181+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2d9810a2a0 con 0x7f2d980f5410 2023-12-06T03:22:01.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.182+0000 7f2d94ff9700 1 -- 172.21.15.175:0/3984150583 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2d8800f300 con 0x7f2d980f5410 2023-12-06T03:22:01.183 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:01.183 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2d8402ff60 msgr2=0x7f2d84032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2d8402ff60 0x7f2d84032420 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f2d800096a0 tx=0x7f2d80006b40).stop 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 msgr2=0x7f2d981093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d981093d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2d88014000 tx=0x7f2d8800b040).stop 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 shutdown_connections 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2d8402ff60 0x7f2d84032420 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 --2- 172.21.15.175:0/3984150583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d980f5410 0x7f2d981093d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 >> 172.21.15.175:0/3984150583 conn(0x7f2d980f0b70 msgr2=0x7f2d980f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 shutdown_connections 2023-12-06T03:22:01.185 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:01.184+0000 7f2d9ea43700 1 -- 172.21.15.175:0/3984150583 wait complete. 2023-12-06T03:22:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:00.407663+0000 mgr.smithi055.qjkkyb (mgr.14164) 20 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:01.183001+0000 mon.smithi055 (mon.0) 123 : audit [DBG] from='client.? 172.21.15.175:0/3984150583' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:02.561 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:02.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:03.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 -- 172.21.15.175:0/1437786795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 msgr2=0x7fe39c0f55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 --2- 172.21.15.175:0/1437786795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c0f55e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fe38c009fb0 tx=0x7fe38c009b30).stop 2023-12-06T03:22:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 -- 172.21.15.175:0/1437786795 shutdown_connections 2023-12-06T03:22:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 --2- 172.21.15.175:0/1437786795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c0f55e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 -- 172.21.15.175:0/1437786795 >> 172.21.15.175:0/1437786795 conn(0x7fe39c0f0920 msgr2=0x7fe39c0f2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 -- 172.21.15.175:0/1437786795 shutdown_connections 2023-12-06T03:22:03.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.766+0000 7fe3a1855700 1 -- 172.21.15.175:0/1437786795 wait complete. 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe3a1855700 1 Processor -- start 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe3a1855700 1 -- start start 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe3a1855700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe3a1855700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c060220 con 0x7fe39c0f51c0 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe39affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe39affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39554/0 (socket says 172.21.15.175:39554) 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe39affd700 1 -- 172.21.15.175:0/3662989882 learned_addr learned my addr 172.21.15.175:0/3662989882 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe39affd700 1 -- 172.21.15.175:0/3662989882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c005040 con 0x7fe39c0f51c0 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.767+0000 7fe39affd700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c109260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fe38c013510 tx=0x7fe38c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:03.770 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.768+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe38c009320 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.768+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c060420 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.768+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe38c013f00 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.768+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe38c014d90 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.768+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c109910 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.768+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fe38c014420 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.769+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c059100 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.769+0000 7fe3837fe700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe38402ff60 0x7fe384032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.769+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fe38c0410c0 con 0x7fe39c0f51c0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.769+0000 7fe39a7fc700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe38402ff60 0x7fe384032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.769+0000 7fe39a7fc700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe38402ff60 0x7fe384032420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fe3900096a0 tx=0x7fe390006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:03.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.774+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe38c0095f0 con 0x7fe39c0f51c0 2023-12-06T03:22:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:02.407912+0000 mgr.smithi055.qjkkyb (mgr.14164) 21 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.980+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe39c045ad0 con 0x7fe39c0f51c0 2023-12-06T03:22:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.980+0000 7fe3837fe700 1 -- 172.21.15.175:0/3662989882 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe38c01db70 con 0x7fe39c0f51c0 2023-12-06T03:22:03.981 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:03.982 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:03.983 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe38402ff60 msgr2=0x7fe384032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:03.983 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe38402ff60 0x7fe384032420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fe3900096a0 tx=0x7fe390006b40).stop 2023-12-06T03:22:03.983 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 msgr2=0x7fe39c109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:03.983 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c109260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fe38c013510 tx=0x7fe38c00b040).stop 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 shutdown_connections 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe38402ff60 0x7fe384032420 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 --2- 172.21.15.175:0/3662989882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe39c0f51c0 0x7fe39c109260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 >> 172.21.15.175:0/3662989882 conn(0x7fe39c0f0920 msgr2=0x7fe39c0f14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 shutdown_connections 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:03.982+0000 7fe3a1855700 1 -- 172.21.15.175:0/3662989882 wait complete. 2023-12-06T03:22:03.984 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:05.394 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:05.394 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:03.981338+0000 mon.smithi055 (mon.0) 124 : audit [DBG] from='client.? 172.21.15.175:0/3662989882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:04.408272+0000 mgr.smithi055.qjkkyb (mgr.14164) 22 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- 172.21.15.175:0/3222356600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 msgr2=0x7fd9a80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 --2- 172.21.15.175:0/3222356600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80f55c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd99c009fb0 tx=0x7fd99c009b30).stop 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- 172.21.15.175:0/3222356600 shutdown_connections 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 --2- 172.21.15.175:0/3222356600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80f55c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- 172.21.15.175:0/3222356600 >> 172.21.15.175:0/3222356600 conn(0x7fd9a80f09a0 msgr2=0x7fd9a80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- 172.21.15.175:0/3222356600 shutdown_connections 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- 172.21.15.175:0/3222356600 wait complete. 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 Processor -- start 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- start start 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:06.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.617+0000 7fd9ad936700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80fb5a0 con 0x7fd9a80f51a0 2023-12-06T03:22:06.620 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ac934700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:06.620 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ac934700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:34990/0 (socket says 172.21.15.175:34990) 2023-12-06T03:22:06.620 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ac934700 1 -- 172.21.15.175:0/1891088387 learned_addr learned my addr 172.21.15.175:0/1891088387 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ac934700 1 -- 172.21.15.175:0/1891088387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c005040 con 0x7fd9a80f51a0 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ac934700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80fd120 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd99c013510 tx=0x7fd99c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd99c009320 con 0x7fd9a80f51a0 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a80fb740 con 0x7fd9a80f51a0 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd99c013f00 con 0x7fd9a80f51a0 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd99c014dc0 con 0x7fd9a80f51a0 2023-12-06T03:22:06.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.618+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80fbbd0 con 0x7fd9a80f51a0 2023-12-06T03:22:06.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.619+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd99c01d3d0 con 0x7fd9a80f51a0 2023-12-06T03:22:06.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.619+0000 7fd9a57fa700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd99802ffb0 0x7fd998032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:06.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.619+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd99c041100 con 0x7fd9a80f51a0 2023-12-06T03:22:06.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.619+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8059100 con 0x7fd9a80f51a0 2023-12-06T03:22:06.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.619+0000 7fd9a7fff700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd99802ffb0 0x7fd998032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:06.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.620+0000 7fd9a7fff700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd99802ffb0 0x7fd998032470 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fd9940096a0 tx=0x7fd994006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:06.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.624+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd99c012920 con 0x7fd9a80f51a0 2023-12-06T03:22:06.832 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.830+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd9a80fc2d0 con 0x7fd9a80f51a0 2023-12-06T03:22:06.832 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.831+0000 7fd9a57fa700 1 -- 172.21.15.175:0/1891088387 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd99c02b020 con 0x7fd9a80f51a0 2023-12-06T03:22:06.833 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:06.833 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:06.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd99802ffb0 msgr2=0x7fd998032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:06.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd99802ffb0 0x7fd998032470 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fd9940096a0 tx=0x7fd994006b40).stop 2023-12-06T03:22:06.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 msgr2=0x7fd9a80fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80fd120 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd99c013510 tx=0x7fd99c00b040).stop 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 shutdown_connections 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd99802ffb0 0x7fd998032470 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 --2- 172.21.15.175:0/1891088387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9a80f51a0 0x7fd9a80fd120 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 >> 172.21.15.175:0/1891088387 conn(0x7fd9a80f09a0 msgr2=0x7fd9a805fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 shutdown_connections 2023-12-06T03:22:06.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:06.833+0000 7fd9ad936700 1 -- 172.21.15.175:0/1891088387 wait complete. 2023-12-06T03:22:06.836 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:06.408637+0000 mgr.smithi055.qjkkyb (mgr.14164) 23 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:06.832300+0000 mon.smithi055 (mon.0) 125 : audit [DBG] from='client.? 172.21.15.175:0/1891088387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:08.213 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:08.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:09.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4177506925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 msgr2=0x7fb5d00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 --2- 172.21.15.175:0/4177506925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d00f5830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fb5c0009fb0 tx=0x7fb5c0009b30).stop 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4177506925 shutdown_connections 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 --2- 172.21.15.175:0/4177506925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d00f5830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4177506925 >> 172.21.15.175:0/4177506925 conn(0x7fb5d00f0b70 msgr2=0x7fb5d00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4177506925 shutdown_connections 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.314+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4177506925 wait complete. 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.315+0000 7fb5d78fc700 1 Processor -- start 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.315+0000 7fb5d78fc700 1 -- start start 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.315+0000 7fb5d78fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d0109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.315+0000 7fb5d78fc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0060220 con 0x7fb5d00f5410 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.315+0000 7fb5d5698700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d0109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:09.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5d5698700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d0109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35018/0 (socket says 172.21.15.175:35018) 2023-12-06T03:22:09.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5d5698700 1 -- 172.21.15.175:0/4180983925 learned_addr learned my addr 172.21.15.175:0/4180983925 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5d5698700 1 -- 172.21.15.175:0/4180983925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0005040 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5d5698700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d0109450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fb5c0014000 tx=0x7fb5c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c00095f0 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d0060420 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5c0007e70 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5c0014ea0 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.316+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d0109b00 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.317+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d0059100 con 0x7fb5d00f5410 2023-12-06T03:22:09.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.317+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb5c0025070 con 0x7fb5d00f5410 2023-12-06T03:22:09.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.317+0000 7fb5c5ffb700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb5bc02ff60 0x7fb5bc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:09.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.317+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb5c0040940 con 0x7fb5d00f5410 2023-12-06T03:22:09.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.318+0000 7fb5d4e97700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb5bc02ff60 0x7fb5bc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:09.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.318+0000 7fb5d4e97700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb5bc02ff60 0x7fb5bc032420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fb5cc0096a0 tx=0x7fb5cc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:09.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.322+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb5c00079c0 con 0x7fb5d00f5410 2023-12-06T03:22:09.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.530+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb5d0045ad0 con 0x7fb5d00f5410 2023-12-06T03:22:09.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.531+0000 7fb5c5ffb700 1 -- 172.21.15.175:0/4180983925 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb5c000f300 con 0x7fb5d00f5410 2023-12-06T03:22:09.532 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:09.532 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:09.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb5bc02ff60 msgr2=0x7fb5bc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb5bc02ff60 0x7fb5bc032420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fb5cc0096a0 tx=0x7fb5cc006b40).stop 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 msgr2=0x7fb5d0109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d0109450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fb5c0014000 tx=0x7fb5c000b040).stop 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 shutdown_connections 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb5bc02ff60 0x7fb5bc032420 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 --2- 172.21.15.175:0/4180983925 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5d00f5410 0x7fb5d0109450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 >> 172.21.15.175:0/4180983925 conn(0x7fb5d00f0b70 msgr2=0x7fb5d00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 shutdown_connections 2023-12-06T03:22:09.534 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:09.532+0000 7fb5d78fc700 1 -- 172.21.15.175:0/4180983925 wait complete. 2023-12-06T03:22:09.535 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:08.408973+0000 mgr.smithi055.qjkkyb (mgr.14164) 24 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:10.918 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:10.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:10.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:09.531871+0000 mon.smithi055 (mon.0) 126 : audit [DBG] from='client.? 172.21.15.175:0/4180983925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:10.409360+0000 mgr.smithi055.qjkkyb (mgr.14164) 25 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:13.650 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 -- 172.21.15.175:0/399032606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 msgr2=0x7f48100f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 --2- 172.21.15.175:0/399032606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f48100f5830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f4800009fb0 tx=0x7f4800009b30).stop 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 -- 172.21.15.175:0/399032606 shutdown_connections 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 --2- 172.21.15.175:0/399032606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f48100f5830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 -- 172.21.15.175:0/399032606 >> 172.21.15.175:0/399032606 conn(0x7f48100f0b70 msgr2=0x7f48100f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 -- 172.21.15.175:0/399032606 shutdown_connections 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.647+0000 7f4817e77700 1 -- 172.21.15.175:0/399032606 wait complete. 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4817e77700 1 Processor -- start 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4817e77700 1 -- start start 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4817e77700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f4810106760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4817e77700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4810106c50 con 0x7f48100f5410 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4815c13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f4810106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:13.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4815c13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f4810106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35044/0 (socket says 172.21.15.175:35044) 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4815c13700 1 -- 172.21.15.175:0/48330952 learned_addr learned my addr 172.21.15.175:0/48330952 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.648+0000 7f4815c13700 1 -- 172.21.15.175:0/48330952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800005040 con 0x7f48100f5410 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.649+0000 7f4815c13700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f4810106760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f4800009f80 tx=0x7f480000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.649+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4800009320 con 0x7f48100f5410 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.649+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4810106e50 con 0x7f48100f5410 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.649+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4800013ec0 con 0x7f48100f5410 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.649+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48101072e0 con 0x7f48100f5410 2023-12-06T03:22:13.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.649+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f480001ece0 con 0x7f48100f5410 2023-12-06T03:22:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.650+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4810045ad0 con 0x7f48100f5410 2023-12-06T03:22:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.650+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f480001e420 con 0x7f48100f5410 2023-12-06T03:22:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.650+0000 7f48067fc700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f47fc02ff10 0x7f47fc0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.650+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f480001ee40 con 0x7f48100f5410 2023-12-06T03:22:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.650+0000 7f4815412700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f47fc02ff10 0x7f47fc0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:13.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.651+0000 7f4815412700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f47fc02ff10 0x7f47fc0323d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f480c0096a0 tx=0x7f480c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:13.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.655+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f480001e6f0 con 0x7f48100f5410 2023-12-06T03:22:13.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.858+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f48100fae60 con 0x7f48100f5410 2023-12-06T03:22:13.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.859+0000 7f48067fc700 1 -- 172.21.15.175:0/48330952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f480001cb10 con 0x7f48100f5410 2023-12-06T03:22:13.860 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:13.861 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.860+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f47fc02ff10 msgr2=0x7f47fc0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.860+0000 7f4817e77700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f47fc02ff10 0x7f47fc0323d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f480c0096a0 tx=0x7f480c006b40).stop 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.860+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 msgr2=0x7f4810106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.860+0000 7f4817e77700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f4810106760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f4800009f80 tx=0x7f480000b040).stop 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.861+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 shutdown_connections 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.861+0000 7f4817e77700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f47fc02ff10 0x7f47fc0323d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:13.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.861+0000 7f4817e77700 1 --2- 172.21.15.175:0/48330952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48100f5410 0x7f4810106760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:13.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.861+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 >> 172.21.15.175:0/48330952 conn(0x7f48100f0b70 msgr2=0x7f481005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:13.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.861+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 shutdown_connections 2023-12-06T03:22:13.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:13.861+0000 7f4817e77700 1 -- 172.21.15.175:0/48330952 wait complete. 2023-12-06T03:22:13.863 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:12.409719+0000 mgr.smithi055.qjkkyb (mgr.14164) 26 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:14.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:13.860131+0000 mon.smithi055 (mon.0) 127 : audit [DBG] from='client.? 172.21.15.175:0/48330952' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:15.246 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:15.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:15.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:14.410103+0000 mgr.smithi055.qjkkyb (mgr.14164) 27 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:16.386 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.383+0000 7facc71ef700 1 -- 172.21.15.175:0/194409906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 msgr2=0x7facc00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.383+0000 7facc71ef700 1 --2- 172.21.15.175:0/194409906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc00f5830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7facb4009fb0 tx=0x7facb4009b30).stop 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.383+0000 7facc71ef700 1 -- 172.21.15.175:0/194409906 shutdown_connections 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.383+0000 7facc71ef700 1 --2- 172.21.15.175:0/194409906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc00f5830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.383+0000 7facc71ef700 1 -- 172.21.15.175:0/194409906 >> 172.21.15.175:0/194409906 conn(0x7facc00f0b70 msgr2=0x7facc00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.383+0000 7facc71ef700 1 -- 172.21.15.175:0/194409906 shutdown_connections 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.384+0000 7facc71ef700 1 -- 172.21.15.175:0/194409906 wait complete. 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.384+0000 7facc71ef700 1 Processor -- start 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.384+0000 7facc71ef700 1 -- start start 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.384+0000 7facc71ef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc01093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.384+0000 7facc71ef700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc0060280 con 0x7facc00f5410 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc4f8b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc01093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc4f8b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc01093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59944/0 (socket says 172.21.15.175:59944) 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc4f8b700 1 -- 172.21.15.175:0/225449070 learned_addr learned my addr 172.21.15.175:0/225449070 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc4f8b700 1 -- 172.21.15.175:0/225449070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4005040 con 0x7facc00f5410 2023-12-06T03:22:16.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc4f8b700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc01093f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7facb4013510 tx=0x7facb400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:16.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facb4009320 con 0x7facc00f5410 2023-12-06T03:22:16.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc71ef700 1 -- 172.21.15.175:0/225449070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc0060480 con 0x7facc00f5410 2023-12-06T03:22:16.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7facb4013f00 con 0x7facc00f5410 2023-12-06T03:22:16.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facc71ef700 1 -- 172.21.15.175:0/225449070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc0109b20 con 0x7facc00f5410 2023-12-06T03:22:16.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.385+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facb4014d90 con 0x7facc00f5410 2023-12-06T03:22:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.386+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc0045ad0 con 0x7facc00f5410 2023-12-06T03:22:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.386+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7facb40095f0 con 0x7facc00f5410 2023-12-06T03:22:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.386+0000 7facbd7fa700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7facb002ff60 0x7facb0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.387+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7facb4040fd0 con 0x7facc00f5410 2023-12-06T03:22:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.387+0000 7facbffff700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7facb002ff60 0x7facb0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:16.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.387+0000 7facbffff700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7facb002ff60 0x7facb0032420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7facac0096a0 tx=0x7facac006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:16.393 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.392+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7facb4014420 con 0x7facc00f5410 2023-12-06T03:22:16.600 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.599+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7facc00fae60 con 0x7facc00f5410 2023-12-06T03:22:16.601 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.600+0000 7facbd7fa700 1 -- 172.21.15.175:0/225449070 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7facc00fae60 con 0x7facc00f5410 2023-12-06T03:22:16.601 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:16.601 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7facb002ff60 msgr2=0x7facb0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7facb002ff60 0x7facb0032420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7facac0096a0 tx=0x7facac006b40).stop 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 msgr2=0x7facc01093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc01093f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7facb4013510 tx=0x7facb400b040).stop 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 shutdown_connections 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7facb002ff60 0x7facb0032420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:16.603 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 --2- 172.21.15.175:0/225449070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7facc00f5410 0x7facc01093f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:16.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 >> 172.21.15.175:0/225449070 conn(0x7facc00f0b70 msgr2=0x7facc00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:16.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 shutdown_connections 2023-12-06T03:22:16.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:16.601+0000 7facaaffd700 1 -- 172.21.15.175:0/225449070 wait complete. 2023-12-06T03:22:16.604 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:16.410341+0000 mgr.smithi055.qjkkyb (mgr.14164) 28 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:16.600833+0000 mon.smithi055 (mon.0) 128 : audit [DBG] from='client.? 172.21.15.175:0/225449070' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:17.969 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:17.969 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:19.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:18.410649+0000 mgr.smithi055.qjkkyb (mgr.14164) 29 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:21.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 -- 172.21.15.175:0/216677555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 msgr2=0x7f35880f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:21.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 --2- 172.21.15.175:0/216677555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880f55c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f3570009fb0 tx=0x7f3570009b30).stop 2023-12-06T03:22:21.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 -- 172.21.15.175:0/216677555 shutdown_connections 2023-12-06T03:22:21.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 --2- 172.21.15.175:0/216677555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880f55c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:21.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 -- 172.21.15.175:0/216677555 >> 172.21.15.175:0/216677555 conn(0x7f35880f09a0 msgr2=0x7f35880f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:21.539 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 -- 172.21.15.175:0/216677555 shutdown_connections 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 -- 172.21.15.175:0/216677555 wait complete. 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.402+0000 7f358d535700 1 Processor -- start 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f358d535700 1 -- start start 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f358d535700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f358d535700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880fb5a0 con 0x7f35880f51a0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f3587fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f3587fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59960/0 (socket says 172.21.15.175:59960) 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f3587fff700 1 -- 172.21.15.175:0/2677904359 learned_addr learned my addr 172.21.15.175:0/2677904359 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.403+0000 7f3587fff700 1 -- 172.21.15.175:0/2677904359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570005040 con 0x7f35880f51a0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.404+0000 7f3587fff700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880fd120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f3570013510 tx=0x7f357000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.404+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3570009320 con 0x7f35880f51a0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.404+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35880fb740 con 0x7f35880f51a0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.404+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35880fbbd0 con 0x7f35880f51a0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.404+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3570013f00 con 0x7f35880f51a0 2023-12-06T03:22:21.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.404+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3570014dc0 con 0x7f35880f51a0 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.405+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f35700095f0 con 0x7f35880f51a0 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.405+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588059100 con 0x7f35880f51a0 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.405+0000 7f3584ff9700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f357402ff60 0x7f3574032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.405+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3570041050 con 0x7f35880f51a0 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.405+0000 7f35877fe700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f357402ff60 0x7f3574032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.406+0000 7f35877fe700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f357402ff60 0x7f3574032420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f35780096a0 tx=0x7f3578006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:21.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.410+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3570014420 con 0x7f35880f51a0 2023-12-06T03:22:21.618 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.617+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3588045ad0 con 0x7f35880f51a0 2023-12-06T03:22:21.619 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.618+0000 7f3584ff9700 1 -- 172.21.15.175:0/2677904359 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f357001c920 con 0x7f35880f51a0 2023-12-06T03:22:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:21.619 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f357402ff60 msgr2=0x7f3574032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f357402ff60 0x7f3574032420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f35780096a0 tx=0x7f3578006b40).stop 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 msgr2=0x7f35880fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880fd120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f3570013510 tx=0x7f357000b040).stop 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 shutdown_connections 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f357402ff60 0x7f3574032420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 --2- 172.21.15.175:0/2677904359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35880f51a0 0x7f35880fd120 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 >> 172.21.15.175:0/2677904359 conn(0x7f35880f09a0 msgr2=0x7f358805fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:21.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 shutdown_connections 2023-12-06T03:22:21.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:21.620+0000 7f358d535700 1 -- 172.21.15.175:0/2677904359 wait complete. 2023-12-06T03:22:21.622 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:20.411014+0000 mgr.smithi055.qjkkyb (mgr.14164) 30 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:22.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:21.619116+0000 mon.smithi055 (mon.0) 129 : audit [DBG] from='client.? 172.21.15.175:0/2677904359' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:22.411365+0000 mgr.smithi055.qjkkyb (mgr.14164) 31 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:23.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:22.549290+0000 mon.smithi055 (mon.0) 130 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:22:23.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:22.552955+0000 mon.smithi055 (mon.0) 131 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:22:24.887 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:24.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:25.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:24.411704+0000 mgr.smithi055.qjkkyb (mgr.14164) 32 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 -- 172.21.15.175:0/1627424162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 msgr2=0x7fedac0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 --2- 172.21.15.175:0/1627424162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac0f90f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fed9c009fb0 tx=0x7fed9c009b30).stop 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 -- 172.21.15.175:0/1627424162 shutdown_connections 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 --2- 172.21.15.175:0/1627424162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac0f90f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 -- 172.21.15.175:0/1627424162 >> 172.21.15.175:0/1627424162 conn(0x7fedac0f0b70 msgr2=0x7fedac0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 -- 172.21.15.175:0/1627424162 shutdown_connections 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 -- 172.21.15.175:0/1627424162 wait complete. 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 Processor -- start 2023-12-06T03:22:27.303 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.300+0000 7fedb1a05700 1 -- start start 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedb1a05700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedb1a05700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac108de0 con 0x7fedac060280 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedaaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedaaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56964/0 (socket says 172.21.15.175:56964) 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedaaffd700 1 -- 172.21.15.175:0/1594206006 learned_addr learned my addr 172.21.15.175:0/1594206006 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedaaffd700 1 -- 172.21.15.175:0/1594206006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c005040 con 0x7fedac060280 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedaaffd700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac1088f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fed9c013510 tx=0x7fed9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed9c009320 con 0x7fedac060280 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed9c013f00 con 0x7fedac060280 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac108fe0 con 0x7fedac060280 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed9c014dc0 con 0x7fedac060280 2023-12-06T03:22:27.304 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.301+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac109470 con 0x7fedac060280 2023-12-06T03:22:27.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.302+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fed9c0095f0 con 0x7fedac060280 2023-12-06T03:22:27.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.302+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac045ad0 con 0x7fedac060280 2023-12-06T03:22:27.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.302+0000 7fed937fe700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fed9402ff60 0x7fed94032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:27.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.302+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fed9c041100 con 0x7fedac060280 2023-12-06T03:22:27.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.303+0000 7fedaa7fc700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fed9402ff60 0x7fed94032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:27.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.303+0000 7fedaa7fc700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fed9402ff60 0x7fed94032420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7feda00096a0 tx=0x7feda0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:27.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.307+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fed9c014420 con 0x7fedac060280 2023-12-06T03:22:27.511 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.509+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fedac0fbf50 con 0x7fedac060280 2023-12-06T03:22:27.511 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.510+0000 7fed937fe700 1 -- 172.21.15.175:0/1594206006 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fed9c010230 con 0x7fedac060280 2023-12-06T03:22:27.511 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:27.511 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fed9402ff60 msgr2=0x7fed94032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fed9402ff60 0x7fed94032420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7feda00096a0 tx=0x7feda0006b40).stop 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 msgr2=0x7fedac1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac1088f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fed9c013510 tx=0x7fed9c00b040).stop 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 shutdown_connections 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fed9402ff60 0x7fed94032420 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 --2- 172.21.15.175:0/1594206006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedac060280 0x7fedac1088f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 >> 172.21.15.175:0/1594206006 conn(0x7fedac0f0b70 msgr2=0x7fedac0f1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:27.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 shutdown_connections 2023-12-06T03:22:27.514 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:27.512+0000 7fedb1a05700 1 -- 172.21.15.175:0/1594206006 wait complete. 2023-12-06T03:22:27.514 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:26.411973+0000 mgr.smithi055.qjkkyb (mgr.14164) 33 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:28.902 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:28.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:28.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:27.511379+0000 mon.smithi055 (mon.0) 132 : audit [DBG] from='client.? 172.21.15.175:0/1594206006' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:28.412309+0000 mgr.smithi055.qjkkyb (mgr.14164) 34 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:30.069 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 -- 172.21.15.175:0/2593894878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 msgr2=0x7f3bf40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:30.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 --2- 172.21.15.175:0/2593894878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf40f5830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f3bdc009fb0 tx=0x7f3bdc009b30).stop 2023-12-06T03:22:30.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 -- 172.21.15.175:0/2593894878 shutdown_connections 2023-12-06T03:22:30.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 --2- 172.21.15.175:0/2593894878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf40f5830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:30.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 -- 172.21.15.175:0/2593894878 >> 172.21.15.175:0/2593894878 conn(0x7f3bf40f0b70 msgr2=0x7f3bf40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:30.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 -- 172.21.15.175:0/2593894878 shutdown_connections 2023-12-06T03:22:30.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.068+0000 7f3bf9255700 1 -- 172.21.15.175:0/2593894878 wait complete. 2023-12-06T03:22:30.071 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.069+0000 7f3bf9255700 1 Processor -- start 2023-12-06T03:22:30.071 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.069+0000 7f3bf9255700 1 -- start start 2023-12-06T03:22:30.071 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.069+0000 7f3bf9255700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf4109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.069+0000 7f3bf9255700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4060240 con 0x7f3bf40f5410 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf4109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf4109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56980/0 (socket says 172.21.15.175:56980) 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf2d9d700 1 -- 172.21.15.175:0/1641580401 learned_addr learned my addr 172.21.15.175:0/1641580401 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf2d9d700 1 -- 172.21.15.175:0/1641580401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc005040 con 0x7f3bf40f5410 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf2d9d700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf4109430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f3bdc013510 tx=0x7f3bdc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bdc009320 con 0x7f3bf40f5410 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf4060440 con 0x7f3bf40f5410 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3bdc013f00 con 0x7f3bf40f5410 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3bdc014d90 con 0x7f3bf40f5410 2023-12-06T03:22:30.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.070+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf4109ae0 con 0x7f3bf40f5410 2023-12-06T03:22:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.071+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4045ad0 con 0x7f3bf40f5410 2023-12-06T03:22:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.071+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f3bdc0095f0 con 0x7f3bf40f5410 2023-12-06T03:22:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.071+0000 7f3beb7fe700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3be002ff60 0x7f3be0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.071+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3bdc040fc0 con 0x7f3bf40f5410 2023-12-06T03:22:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.072+0000 7f3bf259c700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3be002ff60 0x7f3be0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:30.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.072+0000 7f3bf259c700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3be002ff60 0x7f3be0032420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3be40096a0 tx=0x7f3be4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:30.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.076+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3bdc014420 con 0x7f3bf40f5410 2023-12-06T03:22:30.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.285+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3bf40fae60 con 0x7f3bf40f5410 2023-12-06T03:22:30.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.286+0000 7f3beb7fe700 1 -- 172.21.15.175:0/1641580401 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3bdc010230 con 0x7f3bf40f5410 2023-12-06T03:22:30.287 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:30.287 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.287+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3be002ff60 msgr2=0x7f3be0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3be002ff60 0x7f3be0032420 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3be40096a0 tx=0x7f3be4006b40).stop 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 msgr2=0x7f3bf4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf4109430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f3bdc013510 tx=0x7f3bdc00b040).stop 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 shutdown_connections 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3be002ff60 0x7f3be0032420 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 --2- 172.21.15.175:0/1641580401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3bf40f5410 0x7f3bf4109430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 >> 172.21.15.175:0/1641580401 conn(0x7f3bf40f0b70 msgr2=0x7f3bf40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:30.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 shutdown_connections 2023-12-06T03:22:30.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:30.288+0000 7f3bf9255700 1 -- 172.21.15.175:0/1641580401 wait complete. 2023-12-06T03:22:30.290 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:30.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:30 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:30.287209+0000 mon.smithi055 (mon.0) 133 : audit [DBG] from='client.? 172.21.15.175:0/1641580401' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:31.679 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:31.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:30.412741+0000 mgr.smithi055.qjkkyb (mgr.14164) 35 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:32.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 -- 172.21.15.175:0/1940406216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 msgr2=0x7f18280f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:32.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 --2- 172.21.15.175:0/1940406216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f18280f6930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f1818009fb0 tx=0x7f1818009b30).stop 2023-12-06T03:22:32.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 -- 172.21.15.175:0/1940406216 shutdown_connections 2023-12-06T03:22:32.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 --2- 172.21.15.175:0/1940406216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f18280f6930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:32.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 -- 172.21.15.175:0/1940406216 >> 172.21.15.175:0/1940406216 conn(0x7f18280f1c70 msgr2=0x7f18280f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:32.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 -- 172.21.15.175:0/1940406216 shutdown_connections 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.825+0000 7f182fe7c700 1 -- 172.21.15.175:0/1940406216 wait complete. 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182fe7c700 1 Processor -- start 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182fe7c700 1 -- start start 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182fe7c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f1828106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182fe7c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828106c20 con 0x7f18280f6510 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182dc18700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f1828106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182dc18700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f1828106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56994/0 (socket says 172.21.15.175:56994) 2023-12-06T03:22:32.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182dc18700 1 -- 172.21.15.175:0/3116966717 learned_addr learned my addr 172.21.15.175:0/3116966717 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:32.829 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182dc18700 1 -- 172.21.15.175:0/3116966717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1818005040 con 0x7f18280f6510 2023-12-06T03:22:32.829 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.826+0000 7f182dc18700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f1828106730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1818013510 tx=0x7f181800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.827+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1818009320 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.827+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1828106e20 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.827+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1818013f00 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.827+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18281072b0 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.827+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1818014d90 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.827+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828045ad0 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.828+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f18180095f0 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.828+0000 7f181e7fc700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f181402ff60 0x7f1814032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.828+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f1818040fc0 con 0x7f18280f6510 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.828+0000 7f182d417700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f181402ff60 0x7f1814032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.828+0000 7f182d417700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f181402ff60 0x7f1814032420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f18240096a0 tx=0x7f1824006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:32.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:32.832+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1818010230 con 0x7f18280f6510 2023-12-06T03:22:33.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.039+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f18280fbf60 con 0x7f18280f6510 2023-12-06T03:22:33.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.039+0000 7f181e7fc700 1 -- 172.21.15.175:0/3116966717 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1818010230 con 0x7f18280f6510 2023-12-06T03:22:33.041 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:33.041 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:33.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f181402ff60 msgr2=0x7f1814032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f181402ff60 0x7f1814032420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f18240096a0 tx=0x7f1824006b40).stop 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 msgr2=0x7f1828106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f1828106730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f1818013510 tx=0x7f181800b040).stop 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 shutdown_connections 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f181402ff60 0x7f1814032420 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 --2- 172.21.15.175:0/3116966717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18280f6510 0x7f1828106730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 >> 172.21.15.175:0/3116966717 conn(0x7f18280f1c70 msgr2=0x7f18280f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 shutdown_connections 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:33.041+0000 7f182fe7c700 1 -- 172.21.15.175:0/3116966717 wait complete. 2023-12-06T03:22:33.043 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:32.413007+0000 mgr.smithi055.qjkkyb (mgr.14164) 36 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:33.040594+0000 mon.smithi055 (mon.0) 134 : audit [DBG] from='client.? 172.21.15.175:0/3116966717' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:34.432 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:34.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:34 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:34.413350+0000 mgr.smithi055.qjkkyb (mgr.14164) 37 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.570+0000 7f60cbfff700 1 -- 172.21.15.175:0/35602239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 msgr2=0x7f60cc0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.570+0000 7f60cbfff700 1 --2- 172.21.15.175:0/35602239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc0f55c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f60bc009fb0 tx=0x7f60bc009b30).stop 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.570+0000 7f60cbfff700 1 -- 172.21.15.175:0/35602239 shutdown_connections 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.570+0000 7f60cbfff700 1 --2- 172.21.15.175:0/35602239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc0f55c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.570+0000 7f60cbfff700 1 -- 172.21.15.175:0/35602239 >> 172.21.15.175:0/35602239 conn(0x7f60cc0f09a0 msgr2=0x7f60cc0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.571+0000 7f60cbfff700 1 -- 172.21.15.175:0/35602239 shutdown_connections 2023-12-06T03:22:35.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.571+0000 7f60cbfff700 1 -- 172.21.15.175:0/35602239 wait complete. 2023-12-06T03:22:35.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.571+0000 7f60cbfff700 1 Processor -- start 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.571+0000 7f60cbfff700 1 -- start start 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.571+0000 7f60cbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc1064c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.571+0000 7f60cbfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc1069b0 con 0x7f60cc0f51a0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60caffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc1064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60caffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc1064c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39518/0 (socket says 172.21.15.175:39518) 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60caffd700 1 -- 172.21.15.175:0/3418243580 learned_addr learned my addr 172.21.15.175:0/3418243580 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60caffd700 1 -- 172.21.15.175:0/3418243580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc005040 con 0x7f60cc0f51a0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60caffd700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc1064c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f60bc013510 tx=0x7f60bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60bc009320 con 0x7f60cc0f51a0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc106bb0 con 0x7f60cc0f51a0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60bc013f00 con 0x7f60cc0f51a0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60bc014dc0 con 0x7f60cc0f51a0 2023-12-06T03:22:35.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.572+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc107040 con 0x7f60cc0f51a0 2023-12-06T03:22:35.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.573+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f60bc0095f0 con 0x7f60cc0f51a0 2023-12-06T03:22:35.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.573+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc045ad0 con 0x7f60cc0f51a0 2023-12-06T03:22:35.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.573+0000 7f60b37fe700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60b402ff60 0x7f60b4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:35.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.573+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f60bc041100 con 0x7f60cc0f51a0 2023-12-06T03:22:35.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.573+0000 7f60ca7fc700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60b402ff60 0x7f60b4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:35.576 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.574+0000 7f60ca7fc700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60b402ff60 0x7f60b4032420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f60c00096a0 tx=0x7f60c0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:35.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.578+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f60bc014420 con 0x7f60cc0f51a0 2023-12-06T03:22:35.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.784+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f60cc107ef0 con 0x7f60cc0f51a0 2023-12-06T03:22:35.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.784+0000 7f60b37fe700 1 -- 172.21.15.175:0/3418243580 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f60bc010230 con 0x7f60cc0f51a0 2023-12-06T03:22:35.786 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:35.786 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60b402ff60 msgr2=0x7f60b4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60b402ff60 0x7f60b4032420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f60c00096a0 tx=0x7f60c0006b40).stop 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 msgr2=0x7f60cc1064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc1064c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f60bc013510 tx=0x7f60bc00b040).stop 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 shutdown_connections 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60b402ff60 0x7f60b4032420 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 --2- 172.21.15.175:0/3418243580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60cc0f51a0 0x7f60cc1064c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:35.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 >> 172.21.15.175:0/3418243580 conn(0x7f60cc0f09a0 msgr2=0x7f60cc0f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:35.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 shutdown_connections 2023-12-06T03:22:35.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:35.786+0000 7f60cbfff700 1 -- 172.21.15.175:0/3418243580 wait complete. 2023-12-06T03:22:35.788 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:36.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:35.785543+0000 mon.smithi055 (mon.0) 135 : audit [DBG] from='client.? 172.21.15.175:0/3418243580' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:37.151 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:37.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:37.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:36.413614+0000 mgr.smithi055.qjkkyb (mgr.14164) 38 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:38.267 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 -- 172.21.15.175:0/2203348711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 msgr2=0x7f395c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:38.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 --2- 172.21.15.175:0/2203348711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c0f5830 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f3944009fb0 tx=0x7f3944009b30).stop 2023-12-06T03:22:38.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 -- 172.21.15.175:0/2203348711 shutdown_connections 2023-12-06T03:22:38.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 --2- 172.21.15.175:0/2203348711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c0f5830 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:38.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 -- 172.21.15.175:0/2203348711 >> 172.21.15.175:0/2203348711 conn(0x7f395c0f0b70 msgr2=0x7f395c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:38.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 -- 172.21.15.175:0/2203348711 shutdown_connections 2023-12-06T03:22:38.268 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.266+0000 7f39614eb700 1 -- 172.21.15.175:0/2203348711 wait complete. 2023-12-06T03:22:38.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f39614eb700 1 Processor -- start 2023-12-06T03:22:38.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f39614eb700 1 -- start start 2023-12-06T03:22:38.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f39614eb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:38.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f39614eb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c060220 con 0x7f395c0f5410 2023-12-06T03:22:38.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f395affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:38.270 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f395affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39532/0 (socket says 172.21.15.175:39532) 2023-12-06T03:22:38.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.267+0000 7f395affd700 1 -- 172.21.15.175:0/2993439517 learned_addr learned my addr 172.21.15.175:0/2993439517 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:38.271 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f395affd700 1 -- 172.21.15.175:0/2993439517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944005040 con 0x7f395c0f5410 2023-12-06T03:22:38.272 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f395affd700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c109450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f3944014000 tx=0x7f394400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:38.272 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39440095f0 con 0x7f395c0f5410 2023-12-06T03:22:38.272 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c060420 con 0x7f395c0f5410 2023-12-06T03:22:38.272 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3944007e70 con 0x7f395c0f5410 2023-12-06T03:22:38.272 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3944014ea0 con 0x7f395c0f5410 2023-12-06T03:22:38.272 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.268+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c109b00 con 0x7f395c0f5410 2023-12-06T03:22:38.273 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.269+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f3944014690 con 0x7f395c0f5410 2023-12-06T03:22:38.273 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.269+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c059100 con 0x7f395c0f5410 2023-12-06T03:22:38.273 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.269+0000 7f39537fe700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f394802ffb0 0x7f3948032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:38.274 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.269+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f39440408f0 con 0x7f395c0f5410 2023-12-06T03:22:38.274 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.269+0000 7f395a7fc700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f394802ffb0 0x7f3948032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:38.274 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.269+0000 7f395a7fc700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f394802ffb0 0x7f3948032470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f394c0096a0 tx=0x7f394c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:38.275 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.274+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f394400f300 con 0x7f395c0f5410 2023-12-06T03:22:38.482 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.481+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f395c045ad0 con 0x7f395c0f5410 2023-12-06T03:22:38.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.482+0000 7f39537fe700 1 -- 172.21.15.175:0/2993439517 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f394400f300 con 0x7f395c0f5410 2023-12-06T03:22:38.483 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:38.483 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f394802ffb0 msgr2=0x7f3948032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f394802ffb0 0x7f3948032470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f394c0096a0 tx=0x7f394c006b40).stop 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 msgr2=0x7f395c109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c109450 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f3944014000 tx=0x7f394400b040).stop 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 shutdown_connections 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f394802ffb0 0x7f3948032470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 --2- 172.21.15.175:0/2993439517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f395c0f5410 0x7f395c109450 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:38.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.483+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 >> 172.21.15.175:0/2993439517 conn(0x7f395c0f0b70 msgr2=0x7f395c0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:38.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.484+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 shutdown_connections 2023-12-06T03:22:38.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:38.484+0000 7f39614eb700 1 -- 172.21.15.175:0/2993439517 wait complete. 2023-12-06T03:22:38.486 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:39.868 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:39.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:38.413956+0000 mgr.smithi055.qjkkyb (mgr.14164) 39 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:38.482959+0000 mon.smithi055 (mon.0) 136 : audit [DBG] from='client.? 172.21.15.175:0/2993439517' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:41.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 -- 172.21.15.175:0/124508551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 msgr2=0x7f71100f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:41.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 --2- 172.21.15.175:0/124508551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f71100f5830 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f7104009fb0 tx=0x7f7104009b30).stop 2023-12-06T03:22:41.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 -- 172.21.15.175:0/124508551 shutdown_connections 2023-12-06T03:22:41.035 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 --2- 172.21.15.175:0/124508551 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f71100f5830 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 -- 172.21.15.175:0/124508551 >> 172.21.15.175:0/124508551 conn(0x7f71100f0b70 msgr2=0x7f71100f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 -- 172.21.15.175:0/124508551 shutdown_connections 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.033+0000 7f7115a20700 1 -- 172.21.15.175:0/124508551 wait complete. 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.034+0000 7f7115a20700 1 Processor -- start 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.034+0000 7f7115a20700 1 -- start start 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.034+0000 7f7115a20700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f7110109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:41.036 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.034+0000 7f7115a20700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110060240 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.034+0000 7f710effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f7110109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f710effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f7110109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39556/0 (socket says 172.21.15.175:39556) 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f710effd700 1 -- 172.21.15.175:0/2086328766 learned_addr learned my addr 172.21.15.175:0/2086328766 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f710effd700 1 -- 172.21.15.175:0/2086328766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104005040 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f710effd700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f7110109400 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f7104013510 tx=0x7f710400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7104009320 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7110060440 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7104013f00 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7104014d90 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.035+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7110109ab0 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.036+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f71040263a0 con 0x7f71100f5410 2023-12-06T03:22:41.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.036+0000 7f70f77fe700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70f802ff60 0x7f70f8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:41.039 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.036+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f71040410d0 con 0x7f71100f5410 2023-12-06T03:22:41.039 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.036+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7110045ad0 con 0x7f71100f5410 2023-12-06T03:22:41.039 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.036+0000 7f710e7fc700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70f802ff60 0x7f70f8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:41.039 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.037+0000 7f710e7fc700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70f802ff60 0x7f70f8032420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f71000096a0 tx=0x7f7100006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:41.042 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.041+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7104012920 con 0x7f71100f5410 2023-12-06T03:22:41.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.248+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f711010a1c0 con 0x7f71100f5410 2023-12-06T03:22:41.250 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.248+0000 7f70f77fe700 1 -- 172.21.15.175:0/2086328766 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7104012330 con 0x7f71100f5410 2023-12-06T03:22:41.250 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:41.250 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70f802ff60 msgr2=0x7f70f8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70f802ff60 0x7f70f8032420 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f71000096a0 tx=0x7f7100006b40).stop 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 msgr2=0x7f7110109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f7110109400 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f7104013510 tx=0x7f710400b040).stop 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 shutdown_connections 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70f802ff60 0x7f70f8032420 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 --2- 172.21.15.175:0/2086328766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71100f5410 0x7f7110109400 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 >> 172.21.15.175:0/2086328766 conn(0x7f71100f0b70 msgr2=0x7f71100f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.250+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 shutdown_connections 2023-12-06T03:22:41.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:41.251+0000 7f7115a20700 1 -- 172.21.15.175:0/2086328766 wait complete. 2023-12-06T03:22:41.253 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:40.414311+0000 mgr.smithi055.qjkkyb (mgr.14164) 40 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:41.249434+0000 mon.smithi055 (mon.0) 137 : audit [DBG] from='client.? 172.21.15.175:0/2086328766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:42.628 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:42.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:43.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/2932636538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 msgr2=0x7fa6e80f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:43.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 --2- 172.21.15.175:0/2932636538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f90f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa6d8009fb0 tx=0x7fa6d8009b30).stop 2023-12-06T03:22:43.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/2932636538 shutdown_connections 2023-12-06T03:22:43.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 --2- 172.21.15.175:0/2932636538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f90f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:43.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/2932636538 >> 172.21.15.175:0/2932636538 conn(0x7fa6e80f0b70 msgr2=0x7fa6e80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/2932636538 shutdown_connections 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.844+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/2932636538 wait complete. 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6ee3fc700 1 Processor -- start 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6ee3fc700 1 -- start start 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6ee3fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6ee3fc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80f4ff0 con 0x7fa6e8060280 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6e7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6e7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39562/0 (socket says 172.21.15.175:39562) 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.845+0000 7fa6e7fff700 1 -- 172.21.15.175:0/4052687232 learned_addr learned my addr 172.21.15.175:0/4052687232 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6e7fff700 1 -- 172.21.15.175:0/4052687232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8005040 con 0x7fa6e8060280 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6e7fff700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f6b70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fa6d8013510 tx=0x7fa6d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6d8009320 con 0x7fa6e8060280 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e80f5190 con 0x7fa6e8060280 2023-12-06T03:22:43.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa6d8013f00 con 0x7fa6e8060280 2023-12-06T03:22:43.849 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6d8014dc0 con 0x7fa6e8060280 2023-12-06T03:22:43.849 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.846+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80f5620 con 0x7fa6e8060280 2023-12-06T03:22:43.849 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.847+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa6d8014420 con 0x7fa6e8060280 2023-12-06T03:22:43.849 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.847+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e8059100 con 0x7fa6e8060280 2023-12-06T03:22:43.850 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.847+0000 7fa6e4ff9700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa6d002ff60 0x7fa6d0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:43.850 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.847+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa6d8041030 con 0x7fa6e8060280 2023-12-06T03:22:43.850 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.847+0000 7fa6e77fe700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa6d002ff60 0x7fa6d0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:43.850 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.847+0000 7fa6e77fe700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa6d002ff60 0x7fa6d0032420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa6dc0096a0 tx=0x7fa6dc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:43.853 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:43.852+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa6d80095f0 con 0x7fa6e8060280 2023-12-06T03:22:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:42.414714+0000 mgr.smithi055.qjkkyb (mgr.14164) 41 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:44.058 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.057+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa6e80fbf50 con 0x7fa6e8060280 2023-12-06T03:22:44.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.058+0000 7fa6e4ff9700 1 -- 172.21.15.175:0/4052687232 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa6d801cb10 con 0x7fa6e8060280 2023-12-06T03:22:44.059 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:44.059 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa6d002ff60 msgr2=0x7fa6d0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa6d002ff60 0x7fa6d0032420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa6dc0096a0 tx=0x7fa6dc006b40).stop 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 msgr2=0x7fa6e80f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f6b70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fa6d8013510 tx=0x7fa6d800b040).stop 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 shutdown_connections 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa6d002ff60 0x7fa6d0032420 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 --2- 172.21.15.175:0/4052687232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8060280 0x7fa6e80f6b70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 >> 172.21.15.175:0/4052687232 conn(0x7fa6e80f0b70 msgr2=0x7fa6e80f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.059+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 shutdown_connections 2023-12-06T03:22:44.061 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:44.060+0000 7fa6ee3fc700 1 -- 172.21.15.175:0/4052687232 wait complete. 2023-12-06T03:22:44.062 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:45.433 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:45.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:44 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:44.059031+0000 mon.smithi055 (mon.0) 138 : audit [DBG] from='client.? 172.21.15.175:0/4052687232' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:44 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:44.415082+0000 mgr.smithi055.qjkkyb (mgr.14164) 42 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:46.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- 172.21.15.175:0/343985087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 msgr2=0x7ff5840f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 --2- 172.21.15.175:0/343985087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5840f68a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7ff574009fb0 tx=0x7ff574009b30).stop 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- 172.21.15.175:0/343985087 shutdown_connections 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 --2- 172.21.15.175:0/343985087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5840f68a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- 172.21.15.175:0/343985087 >> 172.21.15.175:0/343985087 conn(0x7ff5840f1c00 msgr2=0x7ff5840f4020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- 172.21.15.175:0/343985087 shutdown_connections 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- 172.21.15.175:0/343985087 wait complete. 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 Processor -- start 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- start start 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5841066d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:46.588 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.586+0000 7ff58a25c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584106bc0 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.587+0000 7ff5837fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5841066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.587+0000 7ff5837fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5841066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58342/0 (socket says 172.21.15.175:58342) 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.587+0000 7ff5837fe700 1 -- 172.21.15.175:0/4207542396 learned_addr learned my addr 172.21.15.175:0/4207542396 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.587+0000 7ff5837fe700 1 -- 172.21.15.175:0/4207542396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574005040 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.587+0000 7ff5837fe700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5841066d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ff574013510 tx=0x7ff57400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.587+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff574009320 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.588+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff584106dc0 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.588+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff574013f00 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.588+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff574014d90 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.588+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff584107250 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.588+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff584045ad0 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.589+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff5740095f0 con 0x7ff5840f6480 2023-12-06T03:22:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.589+0000 7ff56bfff700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56c02ff60 0x7ff56c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:46.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.589+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff5740410c0 con 0x7ff5840f6480 2023-12-06T03:22:46.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.589+0000 7ff582ffd700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56c02ff60 0x7ff56c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:46.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.589+0000 7ff582ffd700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56c02ff60 0x7ff56c032420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff5780096a0 tx=0x7ff578006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:46.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.594+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff574014420 con 0x7ff5840f6480 2023-12-06T03:22:46.801 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.800+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff584108100 con 0x7ff5840f6480 2023-12-06T03:22:46.801 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.800+0000 7ff56bfff700 1 -- 172.21.15.175:0/4207542396 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff574010230 con 0x7ff5840f6480 2023-12-06T03:22:46.802 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:46.802 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:46.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56c02ff60 msgr2=0x7ff56c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:46.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56c02ff60 0x7ff56c032420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7ff5780096a0 tx=0x7ff578006b40).stop 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 msgr2=0x7ff5841066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5841066d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7ff574013510 tx=0x7ff57400b040).stop 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 shutdown_connections 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56c02ff60 0x7ff56c032420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 --2- 172.21.15.175:0/4207542396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5840f6480 0x7ff5841066d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 >> 172.21.15.175:0/4207542396 conn(0x7ff5840f1c00 msgr2=0x7ff5840f2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 shutdown_connections 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:46.802+0000 7ff58a25c700 1 -- 172.21.15.175:0/4207542396 wait complete. 2023-12-06T03:22:46.804 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:46.415424+0000 mgr.smithi055.qjkkyb (mgr.14164) 43 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:46.801560+0000 mon.smithi055 (mon.0) 139 : audit [DBG] from='client.? 172.21.15.175:0/4207542396' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:48.273 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:48.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 -- 172.21.15.175:0/2147476263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 msgr2=0x7f30a40f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 --2- 172.21.15.175:0/2147476263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a40f55c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f3094009fb0 tx=0x7f3094009b30).stop 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 -- 172.21.15.175:0/2147476263 shutdown_connections 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 --2- 172.21.15.175:0/2147476263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a40f55c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 -- 172.21.15.175:0/2147476263 >> 172.21.15.175:0/2147476263 conn(0x7f30a40f09a0 msgr2=0x7f30a40f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 -- 172.21.15.175:0/2147476263 shutdown_connections 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.431+0000 7f30ab19e700 1 -- 172.21.15.175:0/2147476263 wait complete. 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30ab19e700 1 Processor -- start 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30ab19e700 1 -- start start 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30ab19e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a41086d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:49.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30ab19e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a4108bc0 con 0x7f30a40f51a0 2023-12-06T03:22:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30aa19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a41086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30aa19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a41086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58370/0 (socket says 172.21.15.175:58370) 2023-12-06T03:22:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.432+0000 7f30aa19c700 1 -- 172.21.15.175:0/2788813263 learned_addr learned my addr 172.21.15.175:0/2788813263 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:49.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f30aa19c700 1 -- 172.21.15.175:0/2788813263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3094005040 con 0x7f30a40f51a0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f30aa19c700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a41086d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3094009830 tx=0x7f309400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3094009320 con 0x7f30a40f51a0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a4108dc0 con 0x7f30a40f51a0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30940136c0 con 0x7f30a40f51a0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f309401ee90 con 0x7f30a40f51a0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.433+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a4109250 con 0x7f30a40f51a0 2023-12-06T03:22:49.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.434+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f309401e420 con 0x7f30a40f51a0 2023-12-06T03:22:49.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.434+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c004fa0 con 0x7f30a40f51a0 2023-12-06T03:22:49.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.434+0000 7f309affd700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f309002ff10 0x7f30900323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:49.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.434+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3094040f10 con 0x7f30a40f51a0 2023-12-06T03:22:49.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.434+0000 7f30a999b700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f309002ff10 0x7f30900323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:49.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.435+0000 7f30a999b700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f309002ff10 0x7f30900323d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f30a00096a0 tx=0x7f30a0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:49.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.439+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f309401c790 con 0x7f30a40f51a0 2023-12-06T03:22:49.649 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.647+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f308c005670 con 0x7f30a40f51a0 2023-12-06T03:22:49.649 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.648+0000 7f309affd700 1 -- 172.21.15.175:0/2788813263 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f309402c2d0 con 0x7f30a40f51a0 2023-12-06T03:22:49.649 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:49.650 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f309002ff10 msgr2=0x7f30900323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f309002ff10 0x7f30900323d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f30a00096a0 tx=0x7f30a0006b40).stop 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 msgr2=0x7f30a41086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a41086d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3094009830 tx=0x7f309400b040).stop 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 shutdown_connections 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f309002ff10 0x7f30900323d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 --2- 172.21.15.175:0/2788813263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30a40f51a0 0x7f30a41086d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 >> 172.21.15.175:0/2788813263 conn(0x7f30a40f09a0 msgr2=0x7f30a405fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 shutdown_connections 2023-12-06T03:22:49.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:49.650+0000 7f30ab19e700 1 -- 172.21.15.175:0/2788813263 wait complete. 2023-12-06T03:22:49.652 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:48.415790+0000 mgr.smithi055.qjkkyb (mgr.14164) 44 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:50.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:50 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:49.649262+0000 mon.smithi055 (mon.0) 140 : audit [DBG] from='client.? 172.21.15.175:0/2788813263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:51.042 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:51.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:51.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:50.416225+0000 mgr.smithi055.qjkkyb (mgr.14164) 45 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 -- 172.21.15.175:0/3641113019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 msgr2=0x7f70fc0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 --2- 172.21.15.175:0/3641113019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f90f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f70ec009fb0 tx=0x7f70ec009b30).stop 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 -- 172.21.15.175:0/3641113019 shutdown_connections 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 --2- 172.21.15.175:0/3641113019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f90f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 -- 172.21.15.175:0/3641113019 >> 172.21.15.175:0/3641113019 conn(0x7f70fc0f0b70 msgr2=0x7f70fc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 -- 172.21.15.175:0/3641113019 shutdown_connections 2023-12-06T03:22:52.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.241+0000 7f7102654700 1 -- 172.21.15.175:0/3641113019 wait complete. 2023-12-06T03:22:52.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f7102654700 1 Processor -- start 2023-12-06T03:22:52.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f7102654700 1 -- start start 2023-12-06T03:22:52.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f7102654700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:52.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f7102654700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc0f4ff0 con 0x7f70fc060280 2023-12-06T03:22:52.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f70fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:52.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f70fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58392/0 (socket says 172.21.15.175:58392) 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.242+0000 7f70fbfff700 1 -- 172.21.15.175:0/3919549710 learned_addr learned my addr 172.21.15.175:0/3919549710 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f70fbfff700 1 -- 172.21.15.175:0/3919549710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec005040 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f70fbfff700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f6b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f70ec013510 tx=0x7f70ec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70ec009320 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc0f5190 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc0f5620 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70ec013f00 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.243+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70ec014dc0 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.244+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f70ec014420 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.244+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc004fa0 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.244+0000 7f70f8ff9700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70e402ff60 0x7f70e4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.244+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f70ec041040 con 0x7f70fc060280 2023-12-06T03:22:52.246 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.244+0000 7f70fb7fe700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70e402ff60 0x7f70e4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:52.247 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.245+0000 7f70fb7fe700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70e402ff60 0x7f70e4032420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f70f00096a0 tx=0x7f70f0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:52.250 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.249+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f70ec0095f0 con 0x7f70fc060280 2023-12-06T03:22:52.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.454+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f70dc005670 con 0x7f70fc060280 2023-12-06T03:22:52.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.454+0000 7f70f8ff9700 1 -- 172.21.15.175:0/3919549710 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f70ec01cb10 con 0x7f70fc060280 2023-12-06T03:22:52.456 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:52.456 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:52.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70e402ff60 msgr2=0x7f70e4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:52.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70e402ff60 0x7f70e4032420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f70f00096a0 tx=0x7f70f0006b40).stop 2023-12-06T03:22:52.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 msgr2=0x7f70fc0f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f6b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f70ec013510 tx=0x7f70ec00b040).stop 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 shutdown_connections 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f70e402ff60 0x7f70e4032420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 --2- 172.21.15.175:0/3919549710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70fc060280 0x7f70fc0f6b70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 >> 172.21.15.175:0/3919549710 conn(0x7f70fc0f0b70 msgr2=0x7f70fc0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 shutdown_connections 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:52.456+0000 7f7102654700 1 -- 172.21.15.175:0/3919549710 wait complete. 2023-12-06T03:22:52.458 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:52.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:52.455545+0000 mon.smithi055 (mon.0) 141 : audit [DBG] from='client.? 172.21.15.175:0/3919549710' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:53.873 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:53.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:52.416614+0000 mgr.smithi055.qjkkyb (mgr.14164) 46 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:55.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 -- 172.21.15.175:0/206429323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 msgr2=0x7ff5740f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:55.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 --2- 172.21.15.175:0/206429323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff5740f5830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7ff564009fb0 tx=0x7ff564009b30).stop 2023-12-06T03:22:55.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 -- 172.21.15.175:0/206429323 shutdown_connections 2023-12-06T03:22:55.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 --2- 172.21.15.175:0/206429323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff5740f5830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 -- 172.21.15.175:0/206429323 >> 172.21.15.175:0/206429323 conn(0x7ff5740f0b70 msgr2=0x7ff5740f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 -- 172.21.15.175:0/206429323 shutdown_connections 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.106+0000 7ff57c172700 1 -- 172.21.15.175:0/206429323 wait complete. 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff57c172700 1 Processor -- start 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff57c172700 1 -- start start 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff57c172700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff574106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff57c172700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574106c20 con 0x7ff5740f5410 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff579f0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff574106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff579f0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff574106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58410/0 (socket says 172.21.15.175:58410) 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.107+0000 7ff579f0e700 1 -- 172.21.15.175:0/3349783215 learned_addr learned my addr 172.21.15.175:0/3349783215 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff579f0e700 1 -- 172.21.15.175:0/3349783215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff564005040 con 0x7ff5740f5410 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff579f0e700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff574106730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7ff564013510 tx=0x7ff56400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:55.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff564009320 con 0x7ff5740f5410 2023-12-06T03:22:55.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574106e20 con 0x7ff5740f5410 2023-12-06T03:22:55.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff564013f00 con 0x7ff5740f5410 2023-12-06T03:22:55.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff564014d90 con 0x7ff5740f5410 2023-12-06T03:22:55.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.108+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5741072b0 con 0x7ff5740f5410 2023-12-06T03:22:55.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.109+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff5640095f0 con 0x7ff5740f5410 2023-12-06T03:22:55.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.109+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff574045ad0 con 0x7ff5740f5410 2023-12-06T03:22:55.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.109+0000 7ff56a7fc700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56002ff60 0x7ff560032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:55.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.109+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff5640410c0 con 0x7ff5740f5410 2023-12-06T03:22:55.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.109+0000 7ff57970d700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56002ff60 0x7ff560032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:55.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.110+0000 7ff57970d700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56002ff60 0x7ff560032420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff5700096a0 tx=0x7ff570006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:55.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.114+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff564014420 con 0x7ff5740f5410 2023-12-06T03:22:55.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.319+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff5740fae60 con 0x7ff5740f5410 2023-12-06T03:22:55.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.320+0000 7ff56a7fc700 1 -- 172.21.15.175:0/3349783215 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff56401d7f0 con 0x7ff5740f5410 2023-12-06T03:22:55.321 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:55.321 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56002ff60 msgr2=0x7ff560032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56002ff60 0x7ff560032420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff5700096a0 tx=0x7ff570006b40).stop 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 msgr2=0x7ff574106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff574106730 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7ff564013510 tx=0x7ff56400b040).stop 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 shutdown_connections 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff56002ff60 0x7ff560032420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 --2- 172.21.15.175:0/3349783215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5740f5410 0x7ff574106730 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.321+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 >> 172.21.15.175:0/3349783215 conn(0x7ff5740f0b70 msgr2=0x7ff5740f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.322+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 shutdown_connections 2023-12-06T03:22:55.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:55.322+0000 7ff57c172700 1 -- 172.21.15.175:0/3349783215 wait complete. 2023-12-06T03:22:55.324 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:54 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:54.416956+0000 mgr.smithi055.qjkkyb (mgr.14164) 47 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:55.320867+0000 mon.smithi055 (mon.0) 142 : audit [DBG] from='client.? 172.21.15.175:0/3349783215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:22:56.717 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:22:56.718 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:22:57.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:56.417320+0000 mgr.smithi055.qjkkyb (mgr.14164) 48 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:58.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.424+0000 7fd73c0c0700 1 -- 172.21.15.175:0/2991022171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 msgr2=0x7fd7340f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:58.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.424+0000 7fd73c0c0700 1 --2- 172.21.15.175:0/2991022171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd7340f5830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd728009fb0 tx=0x7fd728009b30).stop 2023-12-06T03:22:58.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.424+0000 7fd73c0c0700 1 -- 172.21.15.175:0/2991022171 shutdown_connections 2023-12-06T03:22:58.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.424+0000 7fd73c0c0700 1 --2- 172.21.15.175:0/2991022171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd7340f5830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:58.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.424+0000 7fd73c0c0700 1 -- 172.21.15.175:0/2991022171 >> 172.21.15.175:0/2991022171 conn(0x7fd7340f0b70 msgr2=0x7fd7340f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:58.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.424+0000 7fd73c0c0700 1 -- 172.21.15.175:0/2991022171 shutdown_connections 2023-12-06T03:22:58.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.425+0000 7fd73c0c0700 1 -- 172.21.15.175:0/2991022171 wait complete. 2023-12-06T03:22:58.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.425+0000 7fd73c0c0700 1 Processor -- start 2023-12-06T03:22:58.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.425+0000 7fd73c0c0700 1 -- start start 2023-12-06T03:22:58.428 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.425+0000 7fd73c0c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd734105630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:58.428 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.425+0000 7fd73c0c0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734105b20 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd739e5c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd734105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd739e5c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd734105630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35470/0 (socket says 172.21.15.175:35470) 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd739e5c700 1 -- 172.21.15.175:0/1931198173 learned_addr learned my addr 172.21.15.175:0/1931198173 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd739e5c700 1 -- 172.21.15.175:0/1931198173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728005040 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd739e5c700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd734105630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fd728013510 tx=0x7fd72800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd728009320 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd734105d20 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd728013f00 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd728014d90 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.426+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7341061b0 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.427+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd734045ad0 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.427+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd7280095f0 con 0x7fd7340f5410 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.427+0000 7fd7267fc700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd72002ff60 0x7fd720032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:22:58.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.427+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd728041000 con 0x7fd7340f5410 2023-12-06T03:22:58.430 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.427+0000 7fd73965b700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd72002ff60 0x7fd720032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:22:58.430 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.428+0000 7fd73965b700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd72002ff60 0x7fd720032420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd7300096a0 tx=0x7fd730006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:22:58.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.432+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd728014420 con 0x7fd7340f5410 2023-12-06T03:22:58.637 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.636+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd7340fae60 con 0x7fd7340f5410 2023-12-06T03:22:58.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.637+0000 7fd7267fc700 1 -- 172.21.15.175:0/1931198173 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd728010230 con 0x7fd7340f5410 2023-12-06T03:22:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:22:58.638 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:22:58.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd72002ff60 msgr2=0x7fd720032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd72002ff60 0x7fd720032420 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd7300096a0 tx=0x7fd730006b40).stop 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 msgr2=0x7fd734105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd734105630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fd728013510 tx=0x7fd72800b040).stop 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 shutdown_connections 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd72002ff60 0x7fd720032420 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 --2- 172.21.15.175:0/1931198173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7340f5410 0x7fd734105630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 >> 172.21.15.175:0/1931198173 conn(0x7fd7340f0b70 msgr2=0x7fd7340f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 shutdown_connections 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:22:58.638+0000 7fd73c0c0700 1 -- 172.21.15.175:0/1931198173 wait complete. 2023-12-06T03:22:58.640 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:22:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:22:58.417725+0000 mgr.smithi055.qjkkyb (mgr.14164) 49 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:22:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:22:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:22:58.637796+0000 mon.smithi055 (mon.0) 143 : audit [DBG] from='client.? 172.21.15.175:0/1931198173' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:00.035 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:00.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:01.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 -- 172.21.15.175:0/2708636681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 msgr2=0x7fcc7c0f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:01.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 --2- 172.21.15.175:0/2708636681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f4be0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fcc6c009fb0 tx=0x7fcc6c009b30).stop 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 -- 172.21.15.175:0/2708636681 shutdown_connections 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 --2- 172.21.15.175:0/2708636681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f4be0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 -- 172.21.15.175:0/2708636681 >> 172.21.15.175:0/2708636681 conn(0x7fcc7c0f02e0 msgr2=0x7fcc7c0f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 -- 172.21.15.175:0/2708636681 shutdown_connections 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.159+0000 7fcc84318700 1 -- 172.21.15.175:0/2708636681 wait complete. 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc84318700 1 Processor -- start 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc84318700 1 -- start start 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc84318700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc84318700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c0f57f0 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc820b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc820b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35486/0 (socket says 172.21.15.175:35486) 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc820b4700 1 -- 172.21.15.175:0/1074251245 learned_addr learned my addr 172.21.15.175:0/1074251245 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc820b4700 1 -- 172.21.15.175:0/1074251245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c005040 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.160+0000 7fcc820b4700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f7370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fcc6c013510 tx=0x7fcc6c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:01.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc6c009320 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c0f5990 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc6c013f00 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc6c014d90 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c0f5e20 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c059100 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.161+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fcc6c0095f0 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.162+0000 7fcc72ffd700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc6802ff60 0x7fcc68032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.162+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fcc6c040fb0 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.162+0000 7fcc818b3700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc6802ff60 0x7fcc68032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:01.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.162+0000 7fcc818b3700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc6802ff60 0x7fcc68032420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcc780096a0 tx=0x7fcc78006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:01.169 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.166+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcc6c014420 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.375+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc7c045ad0 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.376+0000 7fcc72ffd700 1 -- 172.21.15.175:0/1074251245 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcc6c010230 con 0x7fcc7c0f47c0 2023-12-06T03:23:01.377 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:01.377 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.377+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc6802ff60 msgr2=0x7fcc68032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.377+0000 7fcc84318700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc6802ff60 0x7fcc68032420 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fcc780096a0 tx=0x7fcc78006b40).stop 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.377+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 msgr2=0x7fcc7c0f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.377+0000 7fcc84318700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f7370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fcc6c013510 tx=0x7fcc6c00b040).stop 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.377+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 shutdown_connections 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.378+0000 7fcc84318700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc6802ff60 0x7fcc68032420 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.378+0000 7fcc84318700 1 --2- 172.21.15.175:0/1074251245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc7c0f47c0 0x7fcc7c0f7370 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.378+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 >> 172.21.15.175:0/1074251245 conn(0x7fcc7c0f02e0 msgr2=0x7fcc7c0f0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.378+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 shutdown_connections 2023-12-06T03:23:01.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:01.378+0000 7fcc84318700 1 -- 172.21.15.175:0/1074251245 wait complete. 2023-12-06T03:23:01.380 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:00.418040+0000 mgr.smithi055.qjkkyb (mgr.14164) 50 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:01.376941+0000 mon.smithi055 (mon.0) 144 : audit [DBG] from='client.? 172.21.15.175:0/1074251245' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:02.759 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:02.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:02.418403+0000 mgr.smithi055.qjkkyb (mgr.14164) 51 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/2311053947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 msgr2=0x7f9bf00f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 --2- 172.21.15.175:0/2311053947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf00f8f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9be4009fb0 tx=0x7f9be4009b30).stop 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/2311053947 shutdown_connections 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 --2- 172.21.15.175:0/2311053947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf00f8f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/2311053947 >> 172.21.15.175:0/2311053947 conn(0x7f9bf00f09a0 msgr2=0x7f9bf00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/2311053947 shutdown_connections 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/2311053947 wait complete. 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 Processor -- start 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.976+0000 7f9bf5aaf700 1 -- start start 2023-12-06T03:23:03.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.977+0000 7f9bf5aaf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf0106500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:03.979 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.977+0000 7f9bf5aaf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf01069f0 con 0x7f9bf00601e0 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.977+0000 7f9bf4aad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf0106500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.977+0000 7f9bf4aad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf0106500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35508/0 (socket says 172.21.15.175:35508) 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.977+0000 7f9bf4aad700 1 -- 172.21.15.175:0/3649447265 learned_addr learned my addr 172.21.15.175:0/3649447265 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.977+0000 7f9bf4aad700 1 -- 172.21.15.175:0/3649447265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be4005040 con 0x7f9bf00601e0 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.978+0000 7f9bf4aad700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf0106500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f9be4013510 tx=0x7f9be400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.978+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9be4009320 con 0x7f9bf00601e0 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.978+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf0106bf0 con 0x7f9bf00601e0 2023-12-06T03:23:03.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.978+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9be4013f00 con 0x7f9bf00601e0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.978+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9be4014d90 con 0x7f9bf00601e0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.978+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf0107080 con 0x7f9bf00601e0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.979+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9be4014420 con 0x7f9bf00601e0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.979+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf0045ad0 con 0x7f9bf00601e0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.979+0000 7f9bed7fa700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9be002ff60 0x7f9be0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.979+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9be4040ff0 con 0x7f9bf00601e0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.979+0000 7f9beffff700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9be002ff60 0x7f9be0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:03.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.979+0000 7f9beffff700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9be002ff60 0x7f9be0032420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9bdc0096a0 tx=0x7f9bdc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:03.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:03.984+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9be40095f0 con 0x7f9bf00601e0 2023-12-06T03:23:04.190 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.188+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9bf0107f30 con 0x7f9bf00601e0 2023-12-06T03:23:04.190 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.189+0000 7f9bed7fa700 1 -- 172.21.15.175:0/3649447265 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9be401db70 con 0x7f9bf00601e0 2023-12-06T03:23:04.191 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:04.191 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9be002ff60 msgr2=0x7f9be0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9be002ff60 0x7f9be0032420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9bdc0096a0 tx=0x7f9bdc006b40).stop 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 msgr2=0x7f9bf0106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf0106500 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f9be4013510 tx=0x7f9be400b040).stop 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 shutdown_connections 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9be002ff60 0x7f9be0032420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:04.192 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 --2- 172.21.15.175:0/3649447265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bf00601e0 0x7f9bf0106500 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:04.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 >> 172.21.15.175:0/3649447265 conn(0x7f9bf00f09a0 msgr2=0x7f9bf00f1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:04.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 shutdown_connections 2023-12-06T03:23:04.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:04.191+0000 7f9bf5aaf700 1 -- 172.21.15.175:0/3649447265 wait complete. 2023-12-06T03:23:04.193 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:04.190271+0000 mon.smithi055 (mon.0) 145 : audit [DBG] from='client.? 172.21.15.175:0/3649447265' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:04.418710+0000 mgr.smithi055.qjkkyb (mgr.14164) 52 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:05.612 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:05.613 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:06.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.777+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/352092057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 msgr2=0x7f0a980f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:06.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.777+0000 7f0a9ffbd700 1 --2- 172.21.15.175:0/352092057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a980f6930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f0a8c009fb0 tx=0x7f0a8c009b30).stop 2023-12-06T03:23:06.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.777+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/352092057 shutdown_connections 2023-12-06T03:23:06.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.777+0000 7f0a9ffbd700 1 --2- 172.21.15.175:0/352092057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a980f6930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:06.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.777+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/352092057 >> 172.21.15.175:0/352092057 conn(0x7f0a980f1c70 msgr2=0x7f0a980f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.777+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/352092057 shutdown_connections 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/352092057 wait complete. 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9ffbd700 1 Processor -- start 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9ffbd700 1 -- start start 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9ffbd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a98106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9ffbd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98106c20 con 0x7f0a980f6510 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9dd59700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a98106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9dd59700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a98106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56476/0 (socket says 172.21.15.175:56476) 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.778+0000 7f0a9dd59700 1 -- 172.21.15.175:0/1322299712 learned_addr learned my addr 172.21.15.175:0/1322299712 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a9dd59700 1 -- 172.21.15.175:0/1322299712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c005040 con 0x7f0a980f6510 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a9dd59700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a98106730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f0a8c013510 tx=0x7f0a8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a8c009320 con 0x7f0a980f6510 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98106e20 con 0x7f0a980f6510 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a8c013f00 con 0x7f0a980f6510 2023-12-06T03:23:06.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a8c014d90 con 0x7f0a980f6510 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.779+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981072b0 con 0x7f0a980f6510 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.780+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0a8c0263a0 con 0x7f0a980f6510 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.780+0000 7f0a8a7fc700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0a8402ff60 0x7f0a84032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.780+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98045ad0 con 0x7f0a980f6510 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.780+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0a8c0410e0 con 0x7f0a980f6510 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.780+0000 7f0a9d558700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0a8402ff60 0x7f0a84032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:06.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.781+0000 7f0a9d558700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0a8402ff60 0x7f0a84032420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0a940096a0 tx=0x7f0a94006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:06.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.785+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0a8c012920 con 0x7f0a980f6510 2023-12-06T03:23:06.991 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.989+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0a980fc050 con 0x7f0a980f6510 2023-12-06T03:23:06.991 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.990+0000 7f0a8a7fc700 1 -- 172.21.15.175:0/1322299712 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0a8c012330 con 0x7f0a980f6510 2023-12-06T03:23:06.992 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:06.992 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0a8402ff60 msgr2=0x7f0a84032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0a8402ff60 0x7f0a84032420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f0a940096a0 tx=0x7f0a94006b40).stop 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 msgr2=0x7f0a98106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a98106730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f0a8c013510 tx=0x7f0a8c00b040).stop 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 shutdown_connections 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0a8402ff60 0x7f0a84032420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 --2- 172.21.15.175:0/1322299712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a980f6510 0x7f0a98106730 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 >> 172.21.15.175:0/1322299712 conn(0x7f0a980f1c70 msgr2=0x7f0a980f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 shutdown_connections 2023-12-06T03:23:06.993 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:06.992+0000 7f0a9ffbd700 1 -- 172.21.15.175:0/1322299712 wait complete. 2023-12-06T03:23:06.994 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:06.419005+0000 mgr.smithi055.qjkkyb (mgr.14164) 53 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:07.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:06.991244+0000 mon.smithi055 (mon.0) 146 : audit [DBG] from='client.? 172.21.15.175:0/1322299712' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:08.391 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:08.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:09.578 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 -- 172.21.15.175:0/3395678880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 msgr2=0x7f64580f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 --2- 172.21.15.175:0/3395678880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f64580f5830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f644c009fb0 tx=0x7f644c009b30).stop 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 -- 172.21.15.175:0/3395678880 shutdown_connections 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 --2- 172.21.15.175:0/3395678880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f64580f5830 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 -- 172.21.15.175:0/3395678880 >> 172.21.15.175:0/3395678880 conn(0x7f64580f0b70 msgr2=0x7f64580f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 -- 172.21.15.175:0/3395678880 shutdown_connections 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.576+0000 7f645dd02700 1 -- 172.21.15.175:0/3395678880 wait complete. 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.577+0000 7f645dd02700 1 Processor -- start 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.577+0000 7f645dd02700 1 -- start start 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.577+0000 7f645dd02700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f6458106720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.577+0000 7f645dd02700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458106c10 con 0x7f64580f5410 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.577+0000 7f64577fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f6458106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f64577fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f6458106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56504/0 (socket says 172.21.15.175:56504) 2023-12-06T03:23:09.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f64577fe700 1 -- 172.21.15.175:0/1944288604 learned_addr learned my addr 172.21.15.175:0/1944288604 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f64577fe700 1 -- 172.21.15.175:0/1944288604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c005040 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f64577fe700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f6458106720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f644c013510 tx=0x7f644c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f644c009320 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6458106e10 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f644c013f00 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64581072a0 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.578+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f644c014db0 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.579+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f644c0095f0 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.579+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6458045ad0 con 0x7f64580f5410 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.579+0000 7f643ffff700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6440034370 0x7f6440036830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:09.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.579+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f644c040f80 con 0x7f64580f5410 2023-12-06T03:23:09.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.580+0000 7f6456ffd700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6440034370 0x7f6440036830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:09.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.580+0000 7f6456ffd700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6440034370 0x7f6440036830 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f64480096a0 tx=0x7f6448006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:09.585 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.584+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f644c014420 con 0x7f64580f5410 2023-12-06T03:23:09.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.790+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6458108150 con 0x7f64580f5410 2023-12-06T03:23:09.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.791+0000 7f643ffff700 1 -- 172.21.15.175:0/1944288604 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f644c01c920 con 0x7f64580f5410 2023-12-06T03:23:09.792 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:09.792 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:09.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6440034370 msgr2=0x7f6440036830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6440034370 0x7f6440036830 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f64480096a0 tx=0x7f6448006b40).stop 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 msgr2=0x7f6458106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f6458106720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f644c013510 tx=0x7f644c00b040).stop 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 shutdown_connections 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6440034370 0x7f6440036830 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 --2- 172.21.15.175:0/1944288604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64580f5410 0x7f6458106720 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 >> 172.21.15.175:0/1944288604 conn(0x7f64580f0b70 msgr2=0x7f645805fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 shutdown_connections 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:09.792+0000 7f645dd02700 1 -- 172.21.15.175:0/1944288604 wait complete. 2023-12-06T03:23:09.794 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:08.419312+0000 mgr.smithi055.qjkkyb (mgr.14164) 54 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:10.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:09.791669+0000 mon.smithi055 (mon.0) 147 : audit [DBG] from='client.? 172.21.15.175:0/1944288604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:11.185 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:11.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:10.419680+0000 mgr.smithi055.qjkkyb (mgr.14164) 55 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 -- 172.21.15.175:0/3605715650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 msgr2=0x7f501c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 --2- 172.21.15.175:0/3605715650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c0f5830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f500c009fb0 tx=0x7f500c009b30).stop 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 -- 172.21.15.175:0/3605715650 shutdown_connections 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 --2- 172.21.15.175:0/3605715650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c0f5830 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 -- 172.21.15.175:0/3605715650 >> 172.21.15.175:0/3605715650 conn(0x7f501c0f0b70 msgr2=0x7f501c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 -- 172.21.15.175:0/3605715650 shutdown_connections 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 -- 172.21.15.175:0/3605715650 wait complete. 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 Processor -- start 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.403+0000 7f5021899700 1 -- start start 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f5021899700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c10b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f5021899700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c107210 con 0x7f501c0f5410 2023-12-06T03:23:12.406 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f501affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c10b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f501affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c10b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56526/0 (socket says 172.21.15.175:56526) 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f501affd700 1 -- 172.21.15.175:0/2422026189 learned_addr learned my addr 172.21.15.175:0/2422026189 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f501affd700 1 -- 172.21.15.175:0/2422026189 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c005040 con 0x7f501c0f5410 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f501affd700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c10b820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f500c014000 tx=0x7f500c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f500c0095f0 con 0x7f501c0f5410 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c060220 con 0x7f501c0f5410 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f500c007e70 con 0x7f501c0f5410 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f500c014ea0 con 0x7f501c0f5410 2023-12-06T03:23:12.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.404+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c10bd10 con 0x7f501c0f5410 2023-12-06T03:23:12.408 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.405+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f500c01d4e0 con 0x7f501c0f5410 2023-12-06T03:23:12.408 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.405+0000 7f50037fe700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500402ff60 0x7f5004032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:12.408 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.405+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f500c040960 con 0x7f501c0f5410 2023-12-06T03:23:12.408 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.405+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c045ad0 con 0x7f501c0f5410 2023-12-06T03:23:12.409 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.406+0000 7f501a7fc700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500402ff60 0x7f5004032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:12.409 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.406+0000 7f501a7fc700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500402ff60 0x7f5004032420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f50100096a0 tx=0x7f5010006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:12.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.410+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f500c00f8f0 con 0x7f501c0f5410 2023-12-06T03:23:12.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.613+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f501c10c410 con 0x7f501c0f5410 2023-12-06T03:23:12.615 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f50037fe700 1 -- 172.21.15.175:0/2422026189 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f500c02d090 con 0x7f501c0f5410 2023-12-06T03:23:12.615 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:12.615 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500402ff60 msgr2=0x7f5004032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500402ff60 0x7f5004032420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f50100096a0 tx=0x7f5010006b40).stop 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 msgr2=0x7f501c10b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c10b820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f500c014000 tx=0x7f500c00b040).stop 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 shutdown_connections 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500402ff60 0x7f5004032420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 --2- 172.21.15.175:0/2422026189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f501c0f5410 0x7f501c10b820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 >> 172.21.15.175:0/2422026189 conn(0x7f501c0f0b70 msgr2=0x7f501c0f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 shutdown_connections 2023-12-06T03:23:12.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:12.615+0000 7f5021899700 1 -- 172.21.15.175:0/2422026189 wait complete. 2023-12-06T03:23:12.618 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:12.420061+0000 mgr.smithi055.qjkkyb (mgr.14164) 56 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:12.614957+0000 mon.smithi055 (mon.0) 148 : audit [DBG] from='client.? 172.21.15.175:0/2422026189' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:14.036 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:14.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 -- 172.21.15.175:0/802446854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 msgr2=0x7ffac80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 --2- 172.21.15.175:0/802446854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac80f5830 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7ffabc009fb0 tx=0x7ffabc009b30).stop 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 -- 172.21.15.175:0/802446854 shutdown_connections 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 --2- 172.21.15.175:0/802446854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac80f5830 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 -- 172.21.15.175:0/802446854 >> 172.21.15.175:0/802446854 conn(0x7ffac80f0b70 msgr2=0x7ffac80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 -- 172.21.15.175:0/802446854 shutdown_connections 2023-12-06T03:23:15.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.158+0000 7ffaceccc700 1 -- 172.21.15.175:0/802446854 wait complete. 2023-12-06T03:23:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffaceccc700 1 Processor -- start 2023-12-06T03:23:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffaceccc700 1 -- start start 2023-12-06T03:23:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffaceccc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac81066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:15.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffaceccc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8106bd0 con 0x7ffac80f5410 2023-12-06T03:23:15.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffacca68700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac81066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:15.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffacca68700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac81066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56538/0 (socket says 172.21.15.175:56538) 2023-12-06T03:23:15.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.159+0000 7ffacca68700 1 -- 172.21.15.175:0/2421135044 learned_addr learned my addr 172.21.15.175:0/2421135044 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:15.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffacca68700 1 -- 172.21.15.175:0/2421135044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc005040 con 0x7ffac80f5410 2023-12-06T03:23:15.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffacca68700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac81066e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7ffabc013510 tx=0x7ffabc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffabc009320 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac8106dd0 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffabc013f00 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffabc014d90 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.160+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac8107260 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.161+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ffabc0095f0 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.161+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac8045ad0 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.161+0000 7ffac57fa700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffab802ff60 0x7ffab8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.161+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ffabc041000 con 0x7ffac80f5410 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.161+0000 7ffac7fff700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffab802ff60 0x7ffab8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.162+0000 7ffac7fff700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffab802ff60 0x7ffab8032420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ffab40096a0 tx=0x7ffab4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:15.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.166+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ffabc014420 con 0x7ffac80f5410 2023-12-06T03:23:15.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.372+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffac8108110 con 0x7ffac80f5410 2023-12-06T03:23:15.374 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.373+0000 7ffac57fa700 1 -- 172.21.15.175:0/2421135044 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ffabc01d7f0 con 0x7ffac80f5410 2023-12-06T03:23:15.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:15.375 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffab802ff60 msgr2=0x7ffab8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffab802ff60 0x7ffab8032420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ffab40096a0 tx=0x7ffab4006b40).stop 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 msgr2=0x7ffac81066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac81066e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7ffabc013510 tx=0x7ffabc00b040).stop 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 shutdown_connections 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffab802ff60 0x7ffab8032420 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 --2- 172.21.15.175:0/2421135044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac80f5410 0x7ffac81066e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:15.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 >> 172.21.15.175:0/2421135044 conn(0x7ffac80f0b70 msgr2=0x7ffac805fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:15.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 shutdown_connections 2023-12-06T03:23:15.379 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:15.375+0000 7ffaceccc700 1 -- 172.21.15.175:0/2421135044 wait complete. 2023-12-06T03:23:15.379 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:15.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:14.420403+0000 mgr.smithi055.qjkkyb (mgr.14164) 57 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:16.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:15.374221+0000 mon.smithi055 (mon.0) 149 : audit [DBG] from='client.? 172.21.15.175:0/2421135044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:16.775 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:16.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:17.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:16.420725+0000 mgr.smithi055.qjkkyb (mgr.14164) 58 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:17.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 -- 172.21.15.175:0/4252356027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 msgr2=0x7f93440f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:17.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 --2- 172.21.15.175:0/4252356027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f90f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9338009fb0 tx=0x7f9338009b30).stop 2023-12-06T03:23:17.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 -- 172.21.15.175:0/4252356027 shutdown_connections 2023-12-06T03:23:17.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 --2- 172.21.15.175:0/4252356027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f90f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:17.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 -- 172.21.15.175:0/4252356027 >> 172.21.15.175:0/4252356027 conn(0x7f93440f0b70 msgr2=0x7f93440f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 -- 172.21.15.175:0/4252356027 shutdown_connections 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.937+0000 7f934b105700 1 -- 172.21.15.175:0/4252356027 wait complete. 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f934b105700 1 Processor -- start 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f934b105700 1 -- start start 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f934b105700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f934b105700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93440f4ff0 con 0x7f9344060280 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f9348ea1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f9348ea1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:38050/0 (socket says 172.21.15.175:38050) 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.938+0000 7f9348ea1700 1 -- 172.21.15.175:0/3918049212 learned_addr learned my addr 172.21.15.175:0/3918049212 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f9348ea1700 1 -- 172.21.15.175:0/3918049212 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338005040 con 0x7f9344060280 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f9348ea1700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f6b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f9338013510 tx=0x7f933800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9338009320 con 0x7f9344060280 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93440f5190 con 0x7f9344060280 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9338013f00 con 0x7f9344060280 2023-12-06T03:23:17.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9338014dc0 con 0x7f9344060280 2023-12-06T03:23:17.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.939+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440f5620 con 0x7f9344060280 2023-12-06T03:23:17.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.940+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f933801d3d0 con 0x7f9344060280 2023-12-06T03:23:17.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.940+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344059100 con 0x7f9344060280 2023-12-06T03:23:17.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.940+0000 7f93417fa700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f933402ff60 0x7f9334032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:17.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.940+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9338041100 con 0x7f9344060280 2023-12-06T03:23:17.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.940+0000 7f9343fff700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f933402ff60 0x7f9334032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:17.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.940+0000 7f9343fff700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f933402ff60 0x7f9334032420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f93300096a0 tx=0x7f9330006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:17.946 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:17.945+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9338012510 con 0x7f9344060280 2023-12-06T03:23:18.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.154+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f93440fbf50 con 0x7f9344060280 2023-12-06T03:23:18.156 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.155+0000 7f93417fa700 1 -- 172.21.15.175:0/3918049212 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9338012330 con 0x7f9344060280 2023-12-06T03:23:18.156 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:18.156 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:18.157 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f933402ff60 msgr2=0x7f9334032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f933402ff60 0x7f9334032420 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f93300096a0 tx=0x7f9330006b40).stop 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 msgr2=0x7f93440f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f6b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f9338013510 tx=0x7f933800b040).stop 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 shutdown_connections 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f933402ff60 0x7f9334032420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 --2- 172.21.15.175:0/3918049212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9344060280 0x7f93440f6b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 >> 172.21.15.175:0/3918049212 conn(0x7f93440f0b70 msgr2=0x7f93440f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 shutdown_connections 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:18.156+0000 7f934b105700 1 -- 172.21.15.175:0/3918049212 wait complete. 2023-12-06T03:23:18.158 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:18.701 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:18 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:18.155925+0000 mon.smithi055 (mon.0) 150 : audit [DBG] from='client.? 172.21.15.175:0/3918049212' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:19.528 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:19.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:19.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:18.421007+0000 mgr.smithi055.qjkkyb (mgr.14164) 59 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:20.764 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 -- 172.21.15.175:0/1826133984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 msgr2=0x7f84440f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:20.764 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 --2- 172.21.15.175:0/1826133984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f84440f5830 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f8434009fb0 tx=0x7f8434009b30).stop 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 -- 172.21.15.175:0/1826133984 shutdown_connections 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 --2- 172.21.15.175:0/1826133984 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f84440f5830 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 -- 172.21.15.175:0/1826133984 >> 172.21.15.175:0/1826133984 conn(0x7f84440f0b70 msgr2=0x7f84440f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 -- 172.21.15.175:0/1826133984 shutdown_connections 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 -- 172.21.15.175:0/1826133984 wait complete. 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 Processor -- start 2023-12-06T03:23:20.765 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.763+0000 7f844a727700 1 -- start start 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f844a727700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f8444109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f844a727700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444060240 con 0x7f84440f5410 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8443fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f8444109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8443fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f8444109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:38074/0 (socket says 172.21.15.175:38074) 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8443fff700 1 -- 172.21.15.175:0/2185150124 learned_addr learned my addr 172.21.15.175:0/2185150124 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8443fff700 1 -- 172.21.15.175:0/2185150124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434005040 con 0x7f84440f5410 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8443fff700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f8444109400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8434013510 tx=0x7f843400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8434009320 con 0x7f84440f5410 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8444060440 con 0x7f84440f5410 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8434013f00 con 0x7f84440f5410 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.764+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8434014d90 con 0x7f84440f5410 2023-12-06T03:23:20.767 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.765+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8444109ab0 con 0x7f84440f5410 2023-12-06T03:23:20.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.765+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8434014420 con 0x7f84440f5410 2023-12-06T03:23:20.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.765+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8444045ad0 con 0x7f84440f5410 2023-12-06T03:23:20.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.765+0000 7f8440ff9700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f842c02ff60 0x7f842c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:20.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.765+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8434041080 con 0x7f84440f5410 2023-12-06T03:23:20.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.766+0000 7f84437fe700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f842c02ff60 0x7f842c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:20.768 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.766+0000 7f84437fe700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f842c02ff60 0x7f842c032420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f84380096a0 tx=0x7f8438006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:20.771 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.771+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f84340095f0 con 0x7f84440f5410 2023-12-06T03:23:20.975 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.973+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f84440fae60 con 0x7f84440f5410 2023-12-06T03:23:20.975 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.974+0000 7f8440ff9700 1 -- 172.21.15.175:0/2185150124 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f843401db70 con 0x7f84440f5410 2023-12-06T03:23:20.975 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:20.975 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:20.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f842c02ff60 msgr2=0x7f842c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f842c02ff60 0x7f842c032420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f84380096a0 tx=0x7f8438006b40).stop 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 msgr2=0x7f8444109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f8444109400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8434013510 tx=0x7f843400b040).stop 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 shutdown_connections 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f842c02ff60 0x7f842c032420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 --2- 172.21.15.175:0/2185150124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84440f5410 0x7f8444109400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 >> 172.21.15.175:0/2185150124 conn(0x7f84440f0b70 msgr2=0x7f84440f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 shutdown_connections 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:20.975+0000 7f844a727700 1 -- 172.21.15.175:0/2185150124 wait complete. 2023-12-06T03:23:20.977 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:20.421296+0000 mgr.smithi055.qjkkyb (mgr.14164) 60 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:20.974717+0000 mon.smithi055 (mon.0) 151 : audit [DBG] from='client.? 172.21.15.175:0/2185150124' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:22.385 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:22.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 -- 172.21.15.175:0/1928890792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 msgr2=0x7f0d940f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 --2- 172.21.15.175:0/1928890792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d940f4fa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f0d88009fb0 tx=0x7f0d88009b30).stop 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 -- 172.21.15.175:0/1928890792 shutdown_connections 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 --2- 172.21.15.175:0/1928890792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d940f4fa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 -- 172.21.15.175:0/1928890792 >> 172.21.15.175:0/1928890792 conn(0x7f0d940f02e0 msgr2=0x7f0d940f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 -- 172.21.15.175:0/1928890792 shutdown_connections 2023-12-06T03:23:23.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 -- 172.21.15.175:0/1928890792 wait complete. 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 Processor -- start 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.594+0000 7f0d991cd700 1 -- start start 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d991cd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d94106710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d991cd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94106c00 con 0x7f0d940f4b80 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d92d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d94106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d92d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d94106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:38090/0 (socket says 172.21.15.175:38090) 2023-12-06T03:23:23.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d92d9d700 1 -- 172.21.15.175:0/1744056513 learned_addr learned my addr 172.21.15.175:0/1744056513 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:23.597 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d92d9d700 1 -- 172.21.15.175:0/1744056513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88005040 con 0x7f0d940f4b80 2023-12-06T03:23:23.597 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d92d9d700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d94106710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f0d88013510 tx=0x7f0d8800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d88009320 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d88013f00 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d94106e00 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.595+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0d88014d90 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.596+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d94107290 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.596+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0d880095f0 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.596+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d94045ad0 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.596+0000 7f0d7b7fe700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0d7c02ff60 0x7f0d7c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.596+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0d880410c0 con 0x7f0d940f4b80 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.597+0000 7f0d9259c700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0d7c02ff60 0x7f0d7c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.597+0000 7f0d9259c700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0d7c02ff60 0x7f0d7c032420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0d840096a0 tx=0x7f0d84006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:23.605 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.601+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0d88014420 con 0x7f0d940f4b80 2023-12-06T03:23:23.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.807+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0d940fa5d0 con 0x7f0d940f4b80 2023-12-06T03:23:23.809 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.808+0000 7f0d7b7fe700 1 -- 172.21.15.175:0/1744056513 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0d8801d7f0 con 0x7f0d940f4b80 2023-12-06T03:23:23.809 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:23.809 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:23.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0d7c02ff60 msgr2=0x7f0d7c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0d7c02ff60 0x7f0d7c032420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0d840096a0 tx=0x7f0d84006b40).stop 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 msgr2=0x7f0d94106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d94106710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f0d88013510 tx=0x7f0d8800b040).stop 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 shutdown_connections 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0d7c02ff60 0x7f0d7c032420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 --2- 172.21.15.175:0/1744056513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d940f4b80 0x7f0d94106710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:23.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 >> 172.21.15.175:0/1744056513 conn(0x7f0d940f02e0 msgr2=0x7f0d940f2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:23.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 shutdown_connections 2023-12-06T03:23:23.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:23.810+0000 7f0d991cd700 1 -- 172.21.15.175:0/1744056513 wait complete. 2023-12-06T03:23:23.812 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:22.421594+0000 mgr.smithi055.qjkkyb (mgr.14164) 61 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:22.550459+0000 mon.smithi055 (mon.0) 152 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:23:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:22.553969+0000 mon.smithi055 (mon.0) 153 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:23:24.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:23.808245+0000 mon.smithi055 (mon.0) 154 : audit [DBG] from='client.? 172.21.15.175:0/1744056513' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:25.193 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:25.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:25.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:24.421809+0000 mgr.smithi055.qjkkyb (mgr.14164) 62 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- 172.21.15.175:0/1804853883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 msgr2=0x7feab00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 --2- 172.21.15.175:0/1804853883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab00f5830 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7feaa0009fb0 tx=0x7feaa0009b30).stop 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- 172.21.15.175:0/1804853883 shutdown_connections 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 --2- 172.21.15.175:0/1804853883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab00f5830 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- 172.21.15.175:0/1804853883 >> 172.21.15.175:0/1804853883 conn(0x7feab00f0b70 msgr2=0x7feab00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- 172.21.15.175:0/1804853883 shutdown_connections 2023-12-06T03:23:26.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- 172.21.15.175:0/1804853883 wait complete. 2023-12-06T03:23:26.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 Processor -- start 2023-12-06T03:23:26.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- start start 2023-12-06T03:23:26.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab0109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:26.335 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.332+0000 7feab7d1e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0060220 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab5aba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab0109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab5aba700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab0109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51700/0 (socket says 172.21.15.175:51700) 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab5aba700 1 -- 172.21.15.175:0/527292865 learned_addr learned my addr 172.21.15.175:0/527292865 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab5aba700 1 -- 172.21.15.175:0/527292865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0005040 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab5aba700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab0109450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7feaa0013510 tx=0x7feaa000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feaa0009320 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab0060420 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feaa0013f00 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feaa0014d90 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.333+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab0109b00 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.334+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7feaa0014420 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.334+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0059100 con 0x7feab00f5410 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.334+0000 7feaa67fc700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fea9c02ff60 0x7fea9c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:26.336 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.334+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7feaa0040ff0 con 0x7feab00f5410 2023-12-06T03:23:26.337 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.334+0000 7feab52b9700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fea9c02ff60 0x7fea9c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:26.337 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.335+0000 7feab52b9700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fea9c02ff60 0x7fea9c032420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7feaac0096a0 tx=0x7feaac006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:26.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.339+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7feaa00095f0 con 0x7feab00f5410 2023-12-06T03:23:26.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.548+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feab0045ad0 con 0x7feab00f5410 2023-12-06T03:23:26.550 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.548+0000 7feaa67fc700 1 -- 172.21.15.175:0/527292865 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7feaa001db70 con 0x7feab00f5410 2023-12-06T03:23:26.550 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:26.550 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fea9c02ff60 msgr2=0x7fea9c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fea9c02ff60 0x7fea9c032420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7feaac0096a0 tx=0x7feaac006b40).stop 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 msgr2=0x7feab0109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab0109450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7feaa0013510 tx=0x7feaa000b040).stop 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 shutdown_connections 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fea9c02ff60 0x7fea9c032420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 --2- 172.21.15.175:0/527292865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab00f5410 0x7feab0109450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:26.551 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 >> 172.21.15.175:0/527292865 conn(0x7feab00f0b70 msgr2=0x7feab00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:26.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 shutdown_connections 2023-12-06T03:23:26.552 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:26.550+0000 7feab7d1e700 1 -- 172.21.15.175:0/527292865 wait complete. 2023-12-06T03:23:26.552 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:27.934 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:27.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:26.422100+0000 mgr.smithi055.qjkkyb (mgr.14164) 63 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:26.549541+0000 mon.smithi055 (mon.0) 155 : audit [DBG] from='client.? 172.21.15.175:0/527292865' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 -- 172.21.15.175:0/2722013667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 msgr2=0x7fcb040f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 --2- 172.21.15.175:0/2722013667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb040f6930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fcaf4009fb0 tx=0x7fcaf4009b30).stop 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 -- 172.21.15.175:0/2722013667 shutdown_connections 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 --2- 172.21.15.175:0/2722013667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb040f6930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 -- 172.21.15.175:0/2722013667 >> 172.21.15.175:0/2722013667 conn(0x7fcb040f1c70 msgr2=0x7fcb040f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 -- 172.21.15.175:0/2722013667 shutdown_connections 2023-12-06T03:23:29.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.102+0000 7fcb0c138700 1 -- 172.21.15.175:0/2722013667 wait complete. 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb0c138700 1 Processor -- start 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb0c138700 1 -- start start 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb0c138700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb04106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb0c138700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04106c20 con 0x7fcb040f6510 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb09ed4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb04106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb09ed4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb04106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51720/0 (socket says 172.21.15.175:51720) 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.103+0000 7fcb09ed4700 1 -- 172.21.15.175:0/2875882083 learned_addr learned my addr 172.21.15.175:0/2875882083 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcb09ed4700 1 -- 172.21.15.175:0/2875882083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4005040 con 0x7fcb040f6510 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcb09ed4700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb04106730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fcaf4013510 tx=0x7fcaf400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:29.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcaf4009320 con 0x7fcb040f6510 2023-12-06T03:23:29.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb04106e20 con 0x7fcb040f6510 2023-12-06T03:23:29.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcaf4013f00 con 0x7fcb040f6510 2023-12-06T03:23:29.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcaf4014d90 con 0x7fcb040f6510 2023-12-06T03:23:29.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.104+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb041072b0 con 0x7fcb040f6510 2023-12-06T03:23:29.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.105+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fcaf40263a0 con 0x7fcb040f6510 2023-12-06T03:23:29.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.105+0000 7fcafa7fc700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcaf002ff60 0x7fcaf0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:29.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.105+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fcaf4041020 con 0x7fcb040f6510 2023-12-06T03:23:29.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.105+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb04045ad0 con 0x7fcb040f6510 2023-12-06T03:23:29.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.105+0000 7fcb096d3700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcaf002ff60 0x7fcaf0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:29.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.105+0000 7fcb096d3700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcaf002ff60 0x7fcaf0032420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcb000096a0 tx=0x7fcb00006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:29.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.110+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcaf4012920 con 0x7fcb040f6510 2023-12-06T03:23:29.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.314+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcb040fc050 con 0x7fcb040f6510 2023-12-06T03:23:29.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.315+0000 7fcafa7fc700 1 -- 172.21.15.175:0/2875882083 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcaf402b020 con 0x7fcb040f6510 2023-12-06T03:23:29.316 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:29.316 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:29.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.316+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcaf002ff60 msgr2=0x7fcaf0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:29.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.316+0000 7fcb0c138700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcaf002ff60 0x7fcaf0032420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcb000096a0 tx=0x7fcb00006b40).stop 2023-12-06T03:23:29.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.316+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 msgr2=0x7fcb04106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.316+0000 7fcb0c138700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb04106730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fcaf4013510 tx=0x7fcaf400b040).stop 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.317+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 shutdown_connections 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.317+0000 7fcb0c138700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcaf002ff60 0x7fcaf0032420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.317+0000 7fcb0c138700 1 --2- 172.21.15.175:0/2875882083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb040f6510 0x7fcb04106730 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.317+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 >> 172.21.15.175:0/2875882083 conn(0x7fcb040f1c70 msgr2=0x7fcb040f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.317+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 shutdown_connections 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:29.317+0000 7fcb0c138700 1 -- 172.21.15.175:0/2875882083 wait complete. 2023-12-06T03:23:29.319 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:28.422436+0000 mgr.smithi055.qjkkyb (mgr.14164) 64 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:29.316252+0000 mon.smithi055 (mon.0) 156 : audit [DBG] from='client.? 172.21.15.175:0/2875882083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:30.761 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:30.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 -- 172.21.15.175:0/3667485108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 msgr2=0x7f61980f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 --2- 172.21.15.175:0/3667485108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980f5830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f6188009fb0 tx=0x7f6188009b30).stop 2023-12-06T03:23:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 -- 172.21.15.175:0/3667485108 shutdown_connections 2023-12-06T03:23:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 --2- 172.21.15.175:0/3667485108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980f5830 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 -- 172.21.15.175:0/3667485108 >> 172.21.15.175:0/3667485108 conn(0x7f61980f0b70 msgr2=0x7f61980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:31.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 -- 172.21.15.175:0/3667485108 shutdown_connections 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 -- 172.21.15.175:0/3667485108 wait complete. 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.923+0000 7f619d592700 1 Processor -- start 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f619d592700 1 -- start start 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f619d592700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f619d592700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61980fb820 con 0x7f61980f5410 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f6196ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f6196ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51740/0 (socket says 172.21.15.175:51740) 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f6196ffd700 1 -- 172.21.15.175:0/2679893196 learned_addr learned my addr 172.21.15.175:0/2679893196 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f6196ffd700 1 -- 172.21.15.175:0/2679893196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188005040 con 0x7f61980f5410 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.924+0000 7f6196ffd700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980fd3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6188013510 tx=0x7f618800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.925+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6188009320 con 0x7f61980f5410 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.925+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61980fb9c0 con 0x7f61980f5410 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.925+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6188013f00 con 0x7f61980f5410 2023-12-06T03:23:31.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.925+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6188014d90 con 0x7f61980f5410 2023-12-06T03:23:31.928 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.925+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61980fbe50 con 0x7f61980f5410 2023-12-06T03:23:31.928 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.925+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f61880095f0 con 0x7f61980f5410 2023-12-06T03:23:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.926+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6198059100 con 0x7f61980f5410 2023-12-06T03:23:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.926+0000 7f617f7fe700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f618002ff60 0x7f6180032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.926+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6188041000 con 0x7f61980f5410 2023-12-06T03:23:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.926+0000 7f61967fc700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f618002ff60 0x7f6180032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:31.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.926+0000 7f61967fc700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f618002ff60 0x7f6180032420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f618c0096a0 tx=0x7f618c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:31.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:31.931+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f618801d7f0 con 0x7f61980f5410 2023-12-06T03:23:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:30.422753+0000 mgr.smithi055.qjkkyb (mgr.14164) 65 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:32.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.138+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f61980fae60 con 0x7f61980f5410 2023-12-06T03:23:32.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.138+0000 7f617f7fe700 1 -- 172.21.15.175:0/2679893196 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6188010230 con 0x7f61980f5410 2023-12-06T03:23:32.140 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:32.140 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:32.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f618002ff60 msgr2=0x7f6180032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:32.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f618002ff60 0x7f6180032420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f618c0096a0 tx=0x7f618c006b40).stop 2023-12-06T03:23:32.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 msgr2=0x7f61980fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:32.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980fd3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f6188013510 tx=0x7f618800b040).stop 2023-12-06T03:23:32.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 shutdown_connections 2023-12-06T03:23:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f618002ff60 0x7f6180032420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 --2- 172.21.15.175:0/2679893196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61980f5410 0x7f61980fd3a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 >> 172.21.15.175:0/2679893196 conn(0x7f61980f0b70 msgr2=0x7f619805fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 shutdown_connections 2023-12-06T03:23:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:32.140+0000 7f619d592700 1 -- 172.21.15.175:0/2679893196 wait complete. 2023-12-06T03:23:32.142 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:32.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:32 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:32.139577+0000 mon.smithi055 (mon.0) 157 : audit [DBG] from='client.? 172.21.15.175:0/2679893196' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:33.533 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:33.534 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:32.423082+0000 mgr.smithi055.qjkkyb (mgr.14164) 66 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/3032259116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 msgr2=0x7fd9bc0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 --2- 172.21.15.175:0/3032259116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc0f6930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fd9b0009fb0 tx=0x7fd9b0009b30).stop 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/3032259116 shutdown_connections 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 --2- 172.21.15.175:0/3032259116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc0f6930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/3032259116 >> 172.21.15.175:0/3032259116 conn(0x7fd9bc0f1c70 msgr2=0x7fd9bc0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/3032259116 shutdown_connections 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/3032259116 wait complete. 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.714+0000 7fd9c1ffd700 1 Processor -- start 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9c1ffd700 1 -- start start 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9c1ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9c1ffd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc108e00 con 0x7fd9bc0f6510 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9bb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9bb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51752/0 (socket says 172.21.15.175:51752) 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9bb7fe700 1 -- 172.21.15.175:0/2270299674 learned_addr learned my addr 172.21.15.175:0/2270299674 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9bb7fe700 1 -- 172.21.15.175:0/2270299674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0005040 con 0x7fd9bc0f6510 2023-12-06T03:23:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.715+0000 7fd9bb7fe700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc108910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fd9b0013510 tx=0x7fd9b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.716+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9b0009320 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.716+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc109000 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.716+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9b0013f00 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.716+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9b0014d90 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.716+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc109490 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.716+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd9b00095f0 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.717+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc045ad0 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.717+0000 7fd9a3fff700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd9a402ff60 0x7fd9a4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.717+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd9b0040fe0 con 0x7fd9bc0f6510 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.717+0000 7fd9baffd700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd9a402ff60 0x7fd9a4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.717+0000 7fd9baffd700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd9a402ff60 0x7fd9a4032420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd9ac0096a0 tx=0x7fd9ac006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:34.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.722+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd9b001d7f0 con 0x7fd9bc0f6510 2023-12-06T03:23:34.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.928+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd9bc10a340 con 0x7fd9bc0f6510 2023-12-06T03:23:34.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.928+0000 7fd9a3fff700 1 -- 172.21.15.175:0/2270299674 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd9b0010230 con 0x7fd9bc0f6510 2023-12-06T03:23:34.930 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:34.930 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:34.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd9a402ff60 msgr2=0x7fd9a4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:34.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd9a402ff60 0x7fd9a4032420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd9ac0096a0 tx=0x7fd9ac006b40).stop 2023-12-06T03:23:34.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 msgr2=0x7fd9bc108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc108910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fd9b0013510 tx=0x7fd9b000b040).stop 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 shutdown_connections 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd9a402ff60 0x7fd9a4032420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 --2- 172.21.15.175:0/2270299674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9bc0f6510 0x7fd9bc108910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 >> 172.21.15.175:0/2270299674 conn(0x7fd9bc0f1c70 msgr2=0x7fd9bc0f2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 shutdown_connections 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:34.930+0000 7fd9c1ffd700 1 -- 172.21.15.175:0/2270299674 wait complete. 2023-12-06T03:23:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:34.423497+0000 mgr.smithi055.qjkkyb (mgr.14164) 67 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:34.929435+0000 mon.smithi055 (mon.0) 158 : audit [DBG] from='client.? 172.21.15.175:0/2270299674' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:36.258 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:36.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 -- 172.21.15.175:0/4001361614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 msgr2=0x7ffb140f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 --2- 172.21.15.175:0/4001361614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb140f90f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7ffb08009fb0 tx=0x7ffb08009b30).stop 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 -- 172.21.15.175:0/4001361614 shutdown_connections 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 --2- 172.21.15.175:0/4001361614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb140f90f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 -- 172.21.15.175:0/4001361614 >> 172.21.15.175:0/4001361614 conn(0x7ffb140f0b70 msgr2=0x7ffb140f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 -- 172.21.15.175:0/4001361614 shutdown_connections 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.475+0000 7ffb19785700 1 -- 172.21.15.175:0/4001361614 wait complete. 2023-12-06T03:23:37.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.476+0000 7ffb19785700 1 Processor -- start 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.476+0000 7ffb19785700 1 -- start start 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.476+0000 7ffb19785700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb141066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.476+0000 7ffb19785700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14106bd0 con 0x7ffb14060280 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb141066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb141066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59834/0 (socket says 172.21.15.175:59834) 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb12ffd700 1 -- 172.21.15.175:0/2399681995 learned_addr learned my addr 172.21.15.175:0/2399681995 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb12ffd700 1 -- 172.21.15.175:0/2399681995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08005040 con 0x7ffb14060280 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb12ffd700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb141066e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ffb08009830 tx=0x7ffb0800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb08009320 con 0x7ffb14060280 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb14106dd0 con 0x7ffb14060280 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb080136c0 con 0x7ffb14060280 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb0801ee90 con 0x7ffb14060280 2023-12-06T03:23:37.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.477+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb14107260 con 0x7ffb14060280 2023-12-06T03:23:37.481 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.478+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb14045ad0 con 0x7ffb14060280 2023-12-06T03:23:37.481 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.478+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ffb0801e420 con 0x7ffb14060280 2023-12-06T03:23:37.481 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.478+0000 7ffafb7fe700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffafc02ff10 0x7ffafc0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:37.481 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.479+0000 7ffb127fc700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffafc02ff10 0x7ffafc0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:37.482 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.479+0000 7ffb127fc700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffafc02ff10 0x7ffafc0323d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ffb040096a0 tx=0x7ffb04006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:37.482 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.480+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ffb0801ccf0 con 0x7ffb14060280 2023-12-06T03:23:37.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.484+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ffb080095f0 con 0x7ffb14060280 2023-12-06T03:23:37.688 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.687+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffb14108110 con 0x7ffb14060280 2023-12-06T03:23:37.689 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.688+0000 7ffafb7fe700 1 -- 172.21.15.175:0/2399681995 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ffb0801cb10 con 0x7ffb14060280 2023-12-06T03:23:37.689 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:37.689 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffafc02ff10 msgr2=0x7ffafc0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffafc02ff10 0x7ffafc0323d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ffb040096a0 tx=0x7ffb04006b40).stop 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 msgr2=0x7ffb141066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb141066e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7ffb08009830 tx=0x7ffb0800b040).stop 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 shutdown_connections 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffafc02ff10 0x7ffafc0323d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 --2- 172.21.15.175:0/2399681995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffb14060280 0x7ffb141066e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.689+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 >> 172.21.15.175:0/2399681995 conn(0x7ffb140f0b70 msgr2=0x7ffb140f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.690+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 shutdown_connections 2023-12-06T03:23:37.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:37.690+0000 7ffb19785700 1 -- 172.21.15.175:0/2399681995 wait complete. 2023-12-06T03:23:37.692 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:37.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:36.423829+0000 mgr.smithi055.qjkkyb (mgr.14164) 68 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:38.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:38 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:37.689057+0000 mon.smithi055 (mon.0) 159 : audit [DBG] from='client.? 172.21.15.175:0/2399681995' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:39.116 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:39.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:38.424171+0000 mgr.smithi055.qjkkyb (mgr.14164) 69 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:40.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1260607660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 msgr2=0x7f14dc0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 --2- 172.21.15.175:0/1260607660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc0f6930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f14cc009fb0 tx=0x7f14cc009b30).stop 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1260607660 shutdown_connections 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 --2- 172.21.15.175:0/1260607660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc0f6930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1260607660 >> 172.21.15.175:0/1260607660 conn(0x7f14dc0f1c70 msgr2=0x7f14dc0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1260607660 shutdown_connections 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.307+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1260607660 wait complete. 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e3cf1700 1 Processor -- start 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e3cf1700 1 -- start start 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e3cf1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e3cf1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc106c20 con 0x7f14dc0f6510 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e1a8d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e1a8d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59860/0 (socket says 172.21.15.175:59860) 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.308+0000 7f14e1a8d700 1 -- 172.21.15.175:0/1159155153 learned_addr learned my addr 172.21.15.175:0/1159155153 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14e1a8d700 1 -- 172.21.15.175:0/1159155153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc005040 con 0x7f14dc0f6510 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14e1a8d700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc106730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f14cc013510 tx=0x7f14cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14cc009320 con 0x7f14dc0f6510 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc106e20 con 0x7f14dc0f6510 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14cc013f00 con 0x7f14dc0f6510 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14cc014d90 con 0x7f14dc0f6510 2023-12-06T03:23:40.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.309+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1072b0 con 0x7f14dc0f6510 2023-12-06T03:23:40.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.310+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f14cc0095f0 con 0x7f14dc0f6510 2023-12-06T03:23:40.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.310+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc045ad0 con 0x7f14dc0f6510 2023-12-06T03:23:40.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.310+0000 7f14d27fc700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14c802ff60 0x7f14c8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:40.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.310+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f14cc040ff0 con 0x7f14dc0f6510 2023-12-06T03:23:40.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.310+0000 7f14e128c700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14c802ff60 0x7f14c8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:40.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.311+0000 7f14e128c700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14c802ff60 0x7f14c8032420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f14d80096a0 tx=0x7f14d8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:40.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.315+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f14cc014420 con 0x7f14dc0f6510 2023-12-06T03:23:40.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.520+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f14dc0fbf60 con 0x7f14dc0f6510 2023-12-06T03:23:40.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.521+0000 7f14d27fc700 1 -- 172.21.15.175:0/1159155153 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f14cc01d7f0 con 0x7f14dc0f6510 2023-12-06T03:23:40.522 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:40.522 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:40.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14c802ff60 msgr2=0x7f14c8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:40.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14c802ff60 0x7f14c8032420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f14d80096a0 tx=0x7f14d8006b40).stop 2023-12-06T03:23:40.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 msgr2=0x7f14dc106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc106730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f14cc013510 tx=0x7f14cc00b040).stop 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 shutdown_connections 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14c802ff60 0x7f14c8032420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 --2- 172.21.15.175:0/1159155153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14dc0f6510 0x7f14dc106730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 >> 172.21.15.175:0/1159155153 conn(0x7f14dc0f1c70 msgr2=0x7f14dc0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 shutdown_connections 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:40.523+0000 7f14e3cf1700 1 -- 172.21.15.175:0/1159155153 wait complete. 2023-12-06T03:23:40.525 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:41.910 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:41.910 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:40.424435+0000 mgr.smithi055.qjkkyb (mgr.14164) 70 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:40.521940+0000 mon.smithi055 (mon.0) 160 : audit [DBG] from='client.? 172.21.15.175:0/1159155153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:43.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 -- 172.21.15.175:0/836008046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 msgr2=0x7f15000f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:43.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 --2- 172.21.15.175:0/836008046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f90f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f14f4009fb0 tx=0x7f14f4009b30).stop 2023-12-06T03:23:43.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 -- 172.21.15.175:0/836008046 shutdown_connections 2023-12-06T03:23:43.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 --2- 172.21.15.175:0/836008046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f90f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:43.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 -- 172.21.15.175:0/836008046 >> 172.21.15.175:0/836008046 conn(0x7f15000f0b70 msgr2=0x7f15000f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:43.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 -- 172.21.15.175:0/836008046 shutdown_connections 2023-12-06T03:23:43.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.072+0000 7f15053c2700 1 -- 172.21.15.175:0/836008046 wait complete. 2023-12-06T03:23:43.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f15053c2700 1 Processor -- start 2023-12-06T03:23:43.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f15053c2700 1 -- start start 2023-12-06T03:23:43.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f15053c2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:43.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f15053c2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000f4ff0 con 0x7f1500060280 2023-12-06T03:23:43.076 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f14feffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f14feffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59890/0 (socket says 172.21.15.175:59890) 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f14feffd700 1 -- 172.21.15.175:0/2432242078 learned_addr learned my addr 172.21.15.175:0/2432242078 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f14feffd700 1 -- 172.21.15.175:0/2432242078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4005040 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.073+0000 7f14feffd700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f6b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f14f4013510 tx=0x7f14f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.074+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14f4009320 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.074+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15000f5190 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.074+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14f4013f00 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.074+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14f4014dc0 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.074+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15000f5620 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.075+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f14f40095f0 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.075+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1500059100 con 0x7f1500060280 2023-12-06T03:23:43.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.075+0000 7f14e77fe700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14e802ff60 0x7f14e8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:43.078 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.075+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f14f4041000 con 0x7f1500060280 2023-12-06T03:23:43.078 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.075+0000 7f14fe7fc700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14e802ff60 0x7f14e8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:43.078 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.075+0000 7f14fe7fc700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14e802ff60 0x7f14e8032420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f14f00096a0 tx=0x7f14f0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:43.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.079+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f14f401c790 con 0x7f1500060280 2023-12-06T03:23:43.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.286+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f15000fbf50 con 0x7f1500060280 2023-12-06T03:23:43.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.286+0000 7f14e77fe700 1 -- 172.21.15.175:0/2432242078 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f14f4010230 con 0x7f1500060280 2023-12-06T03:23:43.288 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:43.288 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14e802ff60 msgr2=0x7f14e8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14e802ff60 0x7f14e8032420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f14f00096a0 tx=0x7f14f0006b40).stop 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 msgr2=0x7f15000f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f6b70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f14f4013510 tx=0x7f14f400b040).stop 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 shutdown_connections 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f14e802ff60 0x7f14e8032420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:43.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 --2- 172.21.15.175:0/2432242078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1500060280 0x7f15000f6b70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:43.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 >> 172.21.15.175:0/2432242078 conn(0x7f15000f0b70 msgr2=0x7f15000f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:43.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 shutdown_connections 2023-12-06T03:23:43.290 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:43.288+0000 7f15053c2700 1 -- 172.21.15.175:0/2432242078 wait complete. 2023-12-06T03:23:43.290 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:42.424635+0000 mgr.smithi055.qjkkyb (mgr.14164) 71 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:43.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:43.287430+0000 mon.smithi055 (mon.0) 161 : audit [DBG] from='client.? 172.21.15.175:0/2432242078' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:44.683 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:44.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:44 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:44.424944+0000 mgr.smithi055.qjkkyb (mgr.14164) 72 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:45.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.864+0000 7f612dc4b700 1 -- 172.21.15.175:0/4213076101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 msgr2=0x7f61280f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:45.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.864+0000 7f612dc4b700 1 --2- 172.21.15.175:0/4213076101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61280f6930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f611c009fb0 tx=0x7f611c009b30).stop 2023-12-06T03:23:45.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.864+0000 7f612dc4b700 1 -- 172.21.15.175:0/4213076101 shutdown_connections 2023-12-06T03:23:45.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.864+0000 7f612dc4b700 1 --2- 172.21.15.175:0/4213076101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61280f6930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:45.867 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.864+0000 7f612dc4b700 1 -- 172.21.15.175:0/4213076101 >> 172.21.15.175:0/4213076101 conn(0x7f61280f1c70 msgr2=0x7f61280f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:45.867 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.865+0000 7f612dc4b700 1 -- 172.21.15.175:0/4213076101 shutdown_connections 2023-12-06T03:23:45.867 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.865+0000 7f612dc4b700 1 -- 172.21.15.175:0/4213076101 wait complete. 2023-12-06T03:23:45.867 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.865+0000 7f612dc4b700 1 Processor -- start 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.865+0000 7f612dc4b700 1 -- start start 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f612dc4b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61281088d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f612dc4b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6128108dc0 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f61277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61281088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f61277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61281088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48524/0 (socket says 172.21.15.175:48524) 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f61277fe700 1 -- 172.21.15.175:0/3733833087 learned_addr learned my addr 172.21.15.175:0/3733833087 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f61277fe700 1 -- 172.21.15.175:0/3733833087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c005040 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f61277fe700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61281088d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f611c013510 tx=0x7f611c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f611c009320 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f611c013f00 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6128108fc0 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.866+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f611c014d90 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.867+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6128109450 con 0x7f61280f6510 2023-12-06T03:23:45.869 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.867+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f611c0095f0 con 0x7f61280f6510 2023-12-06T03:23:45.870 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.867+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6128045ad0 con 0x7f61280f6510 2023-12-06T03:23:45.870 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.867+0000 7f610ffff700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f611002ff60 0x7f6110032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:45.870 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.868+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f611c040fc0 con 0x7f61280f6510 2023-12-06T03:23:45.870 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.868+0000 7f6126ffd700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f611002ff60 0x7f6110032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:45.870 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.868+0000 7f6126ffd700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f611002ff60 0x7f6110032420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f61180096a0 tx=0x7f6118006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:45.874 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:45.873+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f611c014420 con 0x7f61280f6510 2023-12-06T03:23:46.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.080+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f612810aa70 con 0x7f61280f6510 2023-12-06T03:23:46.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.080+0000 7f610ffff700 1 -- 172.21.15.175:0/3733833087 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f611c010230 con 0x7f61280f6510 2023-12-06T03:23:46.082 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:46.082 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f611002ff60 msgr2=0x7f6110032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f611002ff60 0x7f6110032420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f61180096a0 tx=0x7f6118006b40).stop 2023-12-06T03:23:46.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 msgr2=0x7f61281088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61281088d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f611c013510 tx=0x7f611c00b040).stop 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 shutdown_connections 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f611002ff60 0x7f6110032420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 --2- 172.21.15.175:0/3733833087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61280f6510 0x7f61281088d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 >> 172.21.15.175:0/3733833087 conn(0x7f61280f1c70 msgr2=0x7f61280f4000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 shutdown_connections 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:46.082+0000 7f612dc4b700 1 -- 172.21.15.175:0/3733833087 wait complete. 2023-12-06T03:23:46.084 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:46.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:46.081596+0000 mon.smithi055 (mon.0) 162 : audit [DBG] from='client.? 172.21.15.175:0/3733833087' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:47.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:46.425307+0000 mgr.smithi055.qjkkyb (mgr.14164) 73 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:47.470 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:47.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 -- 172.21.15.175:0/3304593331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 msgr2=0x7f50240f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 --2- 172.21.15.175:0/3304593331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50240f5830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f5014009fb0 tx=0x7f5014009b30).stop 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 -- 172.21.15.175:0/3304593331 shutdown_connections 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 --2- 172.21.15.175:0/3304593331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50240f5830 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 -- 172.21.15.175:0/3304593331 >> 172.21.15.175:0/3304593331 conn(0x7f50240f0b70 msgr2=0x7f50240f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 -- 172.21.15.175:0/3304593331 shutdown_connections 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.678+0000 7f502a3e7700 1 -- 172.21.15.175:0/3304593331 wait complete. 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.679+0000 7f502a3e7700 1 Processor -- start 2023-12-06T03:23:48.682 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.679+0000 7f502a3e7700 1 -- start start 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.679+0000 7f502a3e7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50241066d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.679+0000 7f502a3e7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024106bc0 con 0x7f50240f5410 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.679+0000 7f5023fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50241066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5023fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50241066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48538/0 (socket says 172.21.15.175:48538) 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5023fff700 1 -- 172.21.15.175:0/3326388848 learned_addr learned my addr 172.21.15.175:0/3326388848 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5023fff700 1 -- 172.21.15.175:0/3326388848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014005040 con 0x7f50240f5410 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5023fff700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50241066d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f5014013510 tx=0x7f501400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5014009320 con 0x7f50240f5410 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5024106dc0 con 0x7f50240f5410 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5014013f00 con 0x7f50240f5410 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5014014d90 con 0x7f50240f5410 2023-12-06T03:23:48.683 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.680+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5024107250 con 0x7f50240f5410 2023-12-06T03:23:48.684 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.681+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f50140095f0 con 0x7f50240f5410 2023-12-06T03:23:48.684 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.681+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5024045ad0 con 0x7f50240f5410 2023-12-06T03:23:48.684 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.681+0000 7f5020ff9700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500c02ff60 0x7f500c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:48.684 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.681+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5014040fe0 con 0x7f50240f5410 2023-12-06T03:23:48.684 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.681+0000 7f50237fe700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500c02ff60 0x7f500c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:48.685 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.682+0000 7f50237fe700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500c02ff60 0x7f500c032420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f50180096a0 tx=0x7f5018006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:48.687 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.686+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5014014420 con 0x7f50240f5410 2023-12-06T03:23:48.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.888+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5024108100 con 0x7f50240f5410 2023-12-06T03:23:48.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.889+0000 7f5020ff9700 1 -- 172.21.15.175:0/3326388848 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5014010230 con 0x7f50240f5410 2023-12-06T03:23:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:48.890 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:48.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500c02ff60 msgr2=0x7f500c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:48.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500c02ff60 0x7f500c032420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f50180096a0 tx=0x7f5018006b40).stop 2023-12-06T03:23:48.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 msgr2=0x7f50241066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50241066d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f5014013510 tx=0x7f501400b040).stop 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 shutdown_connections 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f500c02ff60 0x7f500c032420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 --2- 172.21.15.175:0/3326388848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50240f5410 0x7f50241066d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 >> 172.21.15.175:0/3326388848 conn(0x7f50240f0b70 msgr2=0x7f502405fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 shutdown_connections 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:48.890+0000 7f502a3e7700 1 -- 172.21.15.175:0/3326388848 wait complete. 2023-12-06T03:23:48.892 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:48.425661+0000 mgr.smithi055.qjkkyb (mgr.14164) 74 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:48.889774+0000 mon.smithi055 (mon.0) 163 : audit [DBG] from='client.? 172.21.15.175:0/3326388848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:50.281 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:50.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:51.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 -- 172.21.15.175:0/1117048597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 msgr2=0x7f257c0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 --2- 172.21.15.175:0/1117048597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c0f55c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f256c009fb0 tx=0x7f256c009b30).stop 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 -- 172.21.15.175:0/1117048597 shutdown_connections 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 --2- 172.21.15.175:0/1117048597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c0f55c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 -- 172.21.15.175:0/1117048597 >> 172.21.15.175:0/1117048597 conn(0x7f257c0f09a0 msgr2=0x7f257c0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 -- 172.21.15.175:0/1117048597 shutdown_connections 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 -- 172.21.15.175:0/1117048597 wait complete. 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 Processor -- start 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.445+0000 7f257b59e700 1 -- start start 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c1053b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c1058a0 con 0x7f257c0f51a0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c1053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c1053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48552/0 (socket says 172.21.15.175:48552) 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257a59c700 1 -- 172.21.15.175:0/2718001810 learned_addr learned my addr 172.21.15.175:0/2718001810 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257a59c700 1 -- 172.21.15.175:0/2718001810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c005040 con 0x7f257c0f51a0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257a59c700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c1053b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f256c013510 tx=0x7f256c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f256c009320 con 0x7f257c0f51a0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c105aa0 con 0x7f257c0f51a0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f256c013f00 con 0x7f257c0f51a0 2023-12-06T03:23:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.446+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f256c014dc0 con 0x7f257c0f51a0 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.447+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c105f30 con 0x7f257c0f51a0 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.447+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f256c0095f0 con 0x7f257c0f51a0 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.447+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c045ad0 con 0x7f257c0f51a0 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.447+0000 7f256affd700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f256402ff60 0x7f2564032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.447+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f256c0410c0 con 0x7f257c0f51a0 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.448+0000 7f2579d9b700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f256402ff60 0x7f2564032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.448+0000 7f2579d9b700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f256402ff60 0x7f2564032420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f25700096a0 tx=0x7f2570006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:51.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.453+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f256c01c790 con 0x7f257c0f51a0 2023-12-06T03:23:51.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.659+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f257c106de0 con 0x7f257c0f51a0 2023-12-06T03:23:51.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.659+0000 7f256affd700 1 -- 172.21.15.175:0/2718001810 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f256c010230 con 0x7f257c0f51a0 2023-12-06T03:23:51.662 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:51.662 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f256402ff60 msgr2=0x7f2564032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f256402ff60 0x7f2564032420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f25700096a0 tx=0x7f2570006b40).stop 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 msgr2=0x7f257c1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c1053b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f256c013510 tx=0x7f256c00b040).stop 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 shutdown_connections 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f256402ff60 0x7f2564032420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 --2- 172.21.15.175:0/2718001810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f257c0f51a0 0x7f257c1053b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:51.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 >> 172.21.15.175:0/2718001810 conn(0x7f257c0f09a0 msgr2=0x7f257c0f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:51.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 shutdown_connections 2023-12-06T03:23:51.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:51.661+0000 7f257b59e700 1 -- 172.21.15.175:0/2718001810 wait complete. 2023-12-06T03:23:51.665 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:50.426058+0000 mgr.smithi055.qjkkyb (mgr.14164) 75 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:52.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:51.660581+0000 mon.smithi055 (mon.0) 164 : audit [DBG] from='client.? 172.21.15.175:0/2718001810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:53.058 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:53.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:52.426421+0000 mgr.smithi055.qjkkyb (mgr.14164) 76 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:54.232 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 -- 172.21.15.175:0/2418117252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 msgr2=0x7ff6900f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 --2- 172.21.15.175:0/2418117252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900f57a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff684009fb0 tx=0x7ff684009b30).stop 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 -- 172.21.15.175:0/2418117252 shutdown_connections 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 --2- 172.21.15.175:0/2418117252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900f57a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 -- 172.21.15.175:0/2418117252 >> 172.21.15.175:0/2418117252 conn(0x7ff6900f0b00 msgr2=0x7ff6900f2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 -- 172.21.15.175:0/2418117252 shutdown_connections 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.230+0000 7ff6959e5700 1 -- 172.21.15.175:0/2418117252 wait complete. 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff6959e5700 1 Processor -- start 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff6959e5700 1 -- start start 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff6959e5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff6959e5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6900fb800 con 0x7ff6900f5380 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff68effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff68effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48572/0 (socket says 172.21.15.175:48572) 2023-12-06T03:23:54.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.231+0000 7ff68effd700 1 -- 172.21.15.175:0/77336030 learned_addr learned my addr 172.21.15.175:0/77336030 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff68effd700 1 -- 172.21.15.175:0/77336030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684005040 con 0x7ff6900f5380 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff68effd700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900fd380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff684013510 tx=0x7ff68400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff684009320 con 0x7ff6900f5380 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6900fb9a0 con 0x7ff6900f5380 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff684013f00 con 0x7ff6900f5380 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff684014d90 con 0x7ff6900f5380 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.232+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6900fbe30 con 0x7ff6900f5380 2023-12-06T03:23:54.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.233+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff6840263a0 con 0x7ff6900f5380 2023-12-06T03:23:54.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.233+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff690059100 con 0x7ff6900f5380 2023-12-06T03:23:54.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.233+0000 7ff6777fe700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff67802ff60 0x7ff678032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:54.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.233+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff6840412b0 con 0x7ff6900f5380 2023-12-06T03:23:54.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.233+0000 7ff68e7fc700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff67802ff60 0x7ff678032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:54.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.234+0000 7ff68e7fc700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff67802ff60 0x7ff678032420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff6800096a0 tx=0x7ff680006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:54.239 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.238+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff68401cb10 con 0x7ff6900f5380 2023-12-06T03:23:54.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.446+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff6900fae40 con 0x7ff6900f5380 2023-12-06T03:23:54.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.446+0000 7ff6777fe700 1 -- 172.21.15.175:0/77336030 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff68401cb10 con 0x7ff6900f5380 2023-12-06T03:23:54.448 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:54.449 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:54.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff67802ff60 msgr2=0x7ff678032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:54.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff67802ff60 0x7ff678032420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff6800096a0 tx=0x7ff680006b40).stop 2023-12-06T03:23:54.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 msgr2=0x7ff6900fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:54.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900fd380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff684013510 tx=0x7ff68400b040).stop 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 shutdown_connections 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff67802ff60 0x7ff678032420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 --2- 172.21.15.175:0/77336030 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6900f5380 0x7ff6900fd380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 >> 172.21.15.175:0/77336030 conn(0x7ff6900f0b00 msgr2=0x7ff69005fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 shutdown_connections 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:54.448+0000 7ff6959e5700 1 -- 172.21.15.175:0/77336030 wait complete. 2023-12-06T03:23:54.450 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:54 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:54.426799+0000 mgr.smithi055.qjkkyb (mgr.14164) 77 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:54 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:54.447629+0000 mon.smithi055 (mon.0) 165 : audit [DBG] from='client.? 172.21.15.175:0/77336030' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:55.861 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:55.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:57.037 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.035+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1606762216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 msgr2=0x7ff7a40f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.035+0000 7ff7a8d5f700 1 --2- 172.21.15.175:0/1606762216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a40f90f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7ff78c009fb0 tx=0x7ff78c009b30).stop 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1606762216 shutdown_connections 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 --2- 172.21.15.175:0/1606762216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a40f90f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1606762216 >> 172.21.15.175:0/1606762216 conn(0x7ff7a40f0b70 msgr2=0x7ff7a40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1606762216 shutdown_connections 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1606762216 wait complete. 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 Processor -- start 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 -- start start 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a41066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.036+0000 7ff7a8d5f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4106bd0 con 0x7ff7a4060280 2023-12-06T03:23:57.038 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a41066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:57.039 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a41066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45292/0 (socket says 172.21.15.175:45292) 2023-12-06T03:23:57.039 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a259c700 1 -- 172.21.15.175:0/1336268087 learned_addr learned my addr 172.21.15.175:0/1336268087 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:57.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a259c700 1 -- 172.21.15.175:0/1336268087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c005040 con 0x7ff7a4060280 2023-12-06T03:23:57.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a259c700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a41066e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7ff78c009830 tx=0x7ff78c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:57.040 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff78c009320 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a4106dd0 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff78c0136c0 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff78c01ee90 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.037+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a4107260 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.038+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff78c01e420 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.038+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a4045ad0 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.038+0000 7ff79affd700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff79002ff10 0x7ff7900323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.038+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff78c040fe0 con 0x7ff7a4060280 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.038+0000 7ff7a1d9b700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff79002ff10 0x7ff7900323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:57.041 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.039+0000 7ff7a1d9b700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff79002ff10 0x7ff7900323d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff7940096a0 tx=0x7ff794006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:57.045 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.043+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff78c06d2b0 con 0x7ff7a4060280 2023-12-06T03:23:57.250 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.249+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff7a4108110 con 0x7ff7a4060280 2023-12-06T03:23:57.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.250+0000 7ff79affd700 1 -- 172.21.15.175:0/1336268087 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff78c01c790 con 0x7ff7a4060280 2023-12-06T03:23:57.251 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:23:57.251 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:23:57.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff79002ff10 msgr2=0x7ff7900323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:57.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff79002ff10 0x7ff7900323d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff7940096a0 tx=0x7ff794006b40).stop 2023-12-06T03:23:57.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 msgr2=0x7ff7a41066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:57.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a41066e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7ff78c009830 tx=0x7ff78c00b040).stop 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 shutdown_connections 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff79002ff10 0x7ff7900323d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 --2- 172.21.15.175:0/1336268087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7a4060280 0x7ff7a41066e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 >> 172.21.15.175:0/1336268087 conn(0x7ff7a40f0b70 msgr2=0x7ff7a40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 shutdown_connections 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:57.251+0000 7ff7a8d5f700 1 -- 172.21.15.175:0/1336268087 wait complete. 2023-12-06T03:23:57.253 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:23:57.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:56.427208+0000 mgr.smithi055.qjkkyb (mgr.14164) 78 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:23:57.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:23:57.250801+0000 mon.smithi055 (mon.0) 166 : audit [DBG] from='client.? 172.21.15.175:0/1336268087' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:23:58.682 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:23:58.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 -- 172.21.15.175:0/757536140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 msgr2=0x7fec640f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 --2- 172.21.15.175:0/757536140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec640f5830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fec54009fb0 tx=0x7fec54009b30).stop 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 -- 172.21.15.175:0/757536140 shutdown_connections 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 --2- 172.21.15.175:0/757536140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec640f5830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 -- 172.21.15.175:0/757536140 >> 172.21.15.175:0/757536140 conn(0x7fec640f0b70 msgr2=0x7fec640f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 -- 172.21.15.175:0/757536140 shutdown_connections 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.861+0000 7fec6c6c4700 1 -- 172.21.15.175:0/757536140 wait complete. 2023-12-06T03:23:59.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6c6c4700 1 Processor -- start 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6c6c4700 1 -- start start 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6c6c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec64109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6c6c4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64060240 con 0x7fec640f5410 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6a460700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec64109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6a460700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec64109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45310/0 (socket says 172.21.15.175:45310) 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6a460700 1 -- 172.21.15.175:0/2874694421 learned_addr learned my addr 172.21.15.175:0/2874694421 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6a460700 1 -- 172.21.15.175:0/2874694421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54005040 con 0x7fec640f5410 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.862+0000 7fec6a460700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec64109430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fec54013510 tx=0x7fec5400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.863+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec54009320 con 0x7fec640f5410 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.863+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec64060440 con 0x7fec640f5410 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.863+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fec54013f00 con 0x7fec640f5410 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.863+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec54014d90 con 0x7fec640f5410 2023-12-06T03:23:59.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.863+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64109ae0 con 0x7fec640f5410 2023-12-06T03:23:59.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.863+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fec540263a0 con 0x7fec640f5410 2023-12-06T03:23:59.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.864+0000 7fec5affd700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fec5002ff60 0x7fec50032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:23:59.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.864+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fec54041010 con 0x7fec640f5410 2023-12-06T03:23:59.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.864+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64045ad0 con 0x7fec640f5410 2023-12-06T03:23:59.866 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.864+0000 7fec69c5f700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fec5002ff60 0x7fec50032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:23:59.867 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.864+0000 7fec69c5f700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fec5002ff60 0x7fec50032420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fec600096a0 tx=0x7fec60006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:23:59.870 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:23:59.869+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fec54012920 con 0x7fec640f5410 2023-12-06T03:23:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:23:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:23:58.427450+0000 mgr.smithi055.qjkkyb (mgr.14164) 79 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:00.078 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.077+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fec6410a1c0 con 0x7fec640f5410 2023-12-06T03:24:00.079 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.078+0000 7fec5affd700 1 -- 172.21.15.175:0/2874694421 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fec5402b020 con 0x7fec640f5410 2023-12-06T03:24:00.079 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:00.079 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fec5002ff60 msgr2=0x7fec50032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fec5002ff60 0x7fec50032420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fec600096a0 tx=0x7fec60006b40).stop 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 msgr2=0x7fec64109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec64109430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fec54013510 tx=0x7fec5400b040).stop 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 shutdown_connections 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fec5002ff60 0x7fec50032420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:00.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 --2- 172.21.15.175:0/2874694421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec640f5410 0x7fec64109430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:00.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 >> 172.21.15.175:0/2874694421 conn(0x7fec640f0b70 msgr2=0x7fec640f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:00.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 shutdown_connections 2023-12-06T03:24:00.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:00.080+0000 7fec6c6c4700 1 -- 172.21.15.175:0/2874694421 wait complete. 2023-12-06T03:24:00.082 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:00.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:00 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:00.079259+0000 mon.smithi055 (mon.0) 167 : audit [DBG] from='client.? 172.21.15.175:0/2874694421' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:01.483 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:01.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:00.427783+0000 mgr.smithi055.qjkkyb (mgr.14164) 80 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:02.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/537816083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 msgr2=0x7fb1dc0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:02.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 --2- 172.21.15.175:0/537816083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc0f6930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fb1cc009fb0 tx=0x7fb1cc009b30).stop 2023-12-06T03:24:02.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/537816083 shutdown_connections 2023-12-06T03:24:02.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 --2- 172.21.15.175:0/537816083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc0f6930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:02.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/537816083 >> 172.21.15.175:0/537816083 conn(0x7fb1dc0f1c70 msgr2=0x7fb1dc0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/537816083 shutdown_connections 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.659+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/537816083 wait complete. 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1e0ea3700 1 Processor -- start 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1e0ea3700 1 -- start start 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1e0ea3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1e0ea3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc106c20 con 0x7fb1dc0f6510 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45330/0 (socket says 172.21.15.175:45330) 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1da59c700 1 -- 172.21.15.175:0/1833986577 learned_addr learned my addr 172.21.15.175:0/1833986577 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:02.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1da59c700 1 -- 172.21.15.175:0/1833986577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc005040 con 0x7fb1dc0f6510 2023-12-06T03:24:02.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.660+0000 7fb1da59c700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc106730 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fb1cc013510 tx=0x7fb1cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1cc009320 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb1cc013f00 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc106e20 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb1cc014d90 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc1072b0 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb1cc0095f0 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.661+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc045ad0 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.662+0000 7fb1caffd700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb1c402ff60 0x7fb1c4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.662+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb1cc0410c0 con 0x7fb1dc0f6510 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.662+0000 7fb1d9d9b700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb1c402ff60 0x7fb1c4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:02.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.662+0000 7fb1d9d9b700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb1c402ff60 0x7fb1c4032420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fb1d00096a0 tx=0x7fb1d0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:02.667 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.666+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb1cc01d7f0 con 0x7fb1dc0f6510 2023-12-06T03:24:02.878 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.875+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb1dc0fbf60 con 0x7fb1dc0f6510 2023-12-06T03:24:02.878 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.875+0000 7fb1caffd700 1 -- 172.21.15.175:0/1833986577 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb1cc010230 con 0x7fb1dc0f6510 2023-12-06T03:24:02.878 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:02.878 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:02.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb1c402ff60 msgr2=0x7fb1c4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:02.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb1c402ff60 0x7fb1c4032420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fb1d00096a0 tx=0x7fb1d0006b40).stop 2023-12-06T03:24:02.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 msgr2=0x7fb1dc106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:02.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc106730 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fb1cc013510 tx=0x7fb1cc00b040).stop 2023-12-06T03:24:02.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 shutdown_connections 2023-12-06T03:24:02.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb1c402ff60 0x7fb1c4032420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:02.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 --2- 172.21.15.175:0/1833986577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1dc0f6510 0x7fb1dc106730 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:02.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 >> 172.21.15.175:0/1833986577 conn(0x7fb1dc0f1c70 msgr2=0x7fb1dc0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:02.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 shutdown_connections 2023-12-06T03:24:02.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:02.877+0000 7fb1e0ea3700 1 -- 172.21.15.175:0/1833986577 wait complete. 2023-12-06T03:24:02.880 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:02.428115+0000 mgr.smithi055.qjkkyb (mgr.14164) 81 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:02.876402+0000 mon.smithi055 (mon.0) 168 : audit [DBG] from='client.? 172.21.15.175:0/1833986577' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:04.264 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:04.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:04.428455+0000 mgr.smithi055.qjkkyb (mgr.14164) 82 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 -- 172.21.15.175:0/271303567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 msgr2=0x7f83c80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 --2- 172.21.15.175:0/271303567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c80f55c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f83b8009fb0 tx=0x7f83b8009b30).stop 2023-12-06T03:24:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 -- 172.21.15.175:0/271303567 shutdown_connections 2023-12-06T03:24:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 --2- 172.21.15.175:0/271303567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c80f55c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:05.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 -- 172.21.15.175:0/271303567 >> 172.21.15.175:0/271303567 conn(0x7f83c80f09a0 msgr2=0x7f83c80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 -- 172.21.15.175:0/271303567 shutdown_connections 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.459+0000 7f83cd68e700 1 -- 172.21.15.175:0/271303567 wait complete. 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83cd68e700 1 Processor -- start 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83cd68e700 1 -- start start 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83cd68e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c81091b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83cd68e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c8060240 con 0x7f83c80f51a0 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c81091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c81091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33338/0 (socket says 172.21.15.175:33338) 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.460+0000 7f83c7fff700 1 -- 172.21.15.175:0/3531507105 learned_addr learned my addr 172.21.15.175:0/3531507105 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83c7fff700 1 -- 172.21.15.175:0/3531507105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8005040 con 0x7f83c80f51a0 2023-12-06T03:24:05.464 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83c7fff700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c81091b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f83b8013510 tx=0x7f83b800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:05.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83b8009320 con 0x7f83c80f51a0 2023-12-06T03:24:05.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f83b8013f00 con 0x7f83c80f51a0 2023-12-06T03:24:05.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8060440 con 0x7f83c80f51a0 2023-12-06T03:24:05.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83b8014dc0 con 0x7f83c80f51a0 2023-12-06T03:24:05.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.461+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c8109860 con 0x7f83c80f51a0 2023-12-06T03:24:05.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.462+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f83b80095f0 con 0x7f83c80f51a0 2023-12-06T03:24:05.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.462+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c8045ad0 con 0x7f83c80f51a0 2023-12-06T03:24:05.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.462+0000 7f83c4ff9700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f83b002ff60 0x7f83b0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:05.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.462+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f83b80410c0 con 0x7f83c80f51a0 2023-12-06T03:24:05.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.462+0000 7f83c77fe700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f83b002ff60 0x7f83b0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:05.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.463+0000 7f83c77fe700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f83b002ff60 0x7f83b0032420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f83bc0096a0 tx=0x7f83bc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:05.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.467+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f83b806d2b0 con 0x7f83c80f51a0 2023-12-06T03:24:05.672 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.671+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f83c810a080 con 0x7f83c80f51a0 2023-12-06T03:24:05.673 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.672+0000 7f83c4ff9700 1 -- 172.21.15.175:0/3531507105 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f83b8025350 con 0x7f83c80f51a0 2023-12-06T03:24:05.674 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:05.674 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f83b002ff60 msgr2=0x7f83b0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f83b002ff60 0x7f83b0032420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f83bc0096a0 tx=0x7f83bc006b40).stop 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 msgr2=0x7f83c81091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c81091b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f83b8013510 tx=0x7f83b800b040).stop 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 shutdown_connections 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f83b002ff60 0x7f83b0032420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 --2- 172.21.15.175:0/3531507105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83c80f51a0 0x7f83c81091b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 >> 172.21.15.175:0/3531507105 conn(0x7f83c80f09a0 msgr2=0x7f83c80f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:05.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 shutdown_connections 2023-12-06T03:24:05.676 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:05.674+0000 7f83cd68e700 1 -- 172.21.15.175:0/3531507105 wait complete. 2023-12-06T03:24:05.676 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:06.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:06 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:05.673230+0000 mon.smithi055 (mon.0) 169 : audit [DBG] from='client.? 172.21.15.175:0/3531507105' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:07.076 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:07.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:07.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:06.428805+0000 mgr.smithi055.qjkkyb (mgr.14164) 83 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:08.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 -- 172.21.15.175:0/675272799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 msgr2=0x7f4ef00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:08.241 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 --2- 172.21.15.175:0/675272799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef00f5830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4ee0009fb0 tx=0x7f4ee0009b30).stop 2023-12-06T03:24:08.241 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 -- 172.21.15.175:0/675272799 shutdown_connections 2023-12-06T03:24:08.241 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 --2- 172.21.15.175:0/675272799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef00f5830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:08.241 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 -- 172.21.15.175:0/675272799 >> 172.21.15.175:0/675272799 conn(0x7f4ef00f0b70 msgr2=0x7f4ef00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:08.241 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 -- 172.21.15.175:0/675272799 shutdown_connections 2023-12-06T03:24:08.241 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.239+0000 7f4ef843e700 1 -- 172.21.15.175:0/675272799 wait complete. 2023-12-06T03:24:08.242 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef843e700 1 Processor -- start 2023-12-06T03:24:08.242 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef843e700 1 -- start start 2023-12-06T03:24:08.242 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef843e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef0106710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef843e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0106c00 con 0x7f4ef00f5410 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef61da700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef0106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef61da700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef0106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33354/0 (socket says 172.21.15.175:33354) 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.240+0000 7f4ef61da700 1 -- 172.21.15.175:0/2892072645 learned_addr learned my addr 172.21.15.175:0/2892072645 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ef61da700 1 -- 172.21.15.175:0/2892072645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee0005040 con 0x7f4ef00f5410 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ef61da700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef0106710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4ee0013510 tx=0x7f4ee000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ee0009320 con 0x7f4ef00f5410 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef0106e00 con 0x7f4ef00f5410 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ee0013f00 con 0x7f4ef00f5410 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ee0014d90 con 0x7f4ef00f5410 2023-12-06T03:24:08.243 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.241+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef0107290 con 0x7f4ef00f5410 2023-12-06T03:24:08.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.242+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f4ee00095f0 con 0x7f4ef00f5410 2023-12-06T03:24:08.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.242+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0045ad0 con 0x7f4ef00f5410 2023-12-06T03:24:08.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.242+0000 7f4ee6ffd700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4edc02ff60 0x7f4edc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:08.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.242+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4ee00410c0 con 0x7f4ef00f5410 2023-12-06T03:24:08.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.242+0000 7f4ef59d9700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4edc02ff60 0x7f4edc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:08.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.243+0000 7f4ef59d9700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4edc02ff60 0x7f4edc032420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4eec0096a0 tx=0x7f4eec006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:08.248 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.247+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4ee0014420 con 0x7f4ef00f5410 2023-12-06T03:24:08.451 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.450+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ef00fae60 con 0x7f4ef00f5410 2023-12-06T03:24:08.452 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.451+0000 7f4ee6ffd700 1 -- 172.21.15.175:0/2892072645 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4ee001d7f0 con 0x7f4ef00f5410 2023-12-06T03:24:08.452 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:08.452 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:08.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4edc02ff60 msgr2=0x7f4edc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4edc02ff60 0x7f4edc032420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f4eec0096a0 tx=0x7f4eec006b40).stop 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 msgr2=0x7f4ef0106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef0106710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4ee0013510 tx=0x7f4ee000b040).stop 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 shutdown_connections 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4edc02ff60 0x7f4edc032420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 --2- 172.21.15.175:0/2892072645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ef00f5410 0x7f4ef0106710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 >> 172.21.15.175:0/2892072645 conn(0x7f4ef00f0b70 msgr2=0x7f4ef005fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 shutdown_connections 2023-12-06T03:24:08.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:08.452+0000 7f4ef843e700 1 -- 172.21.15.175:0/2892072645 wait complete. 2023-12-06T03:24:08.455 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:08.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:08 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:08.451733+0000 mon.smithi055 (mon.0) 170 : audit [DBG] from='client.? 172.21.15.175:0/2892072645' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:09.845 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:09.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:08.429208+0000 mgr.smithi055.qjkkyb (mgr.14164) 84 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 -- 172.21.15.175:0/1793528418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 msgr2=0x7f8f7c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 --2- 172.21.15.175:0/1793528418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c0f5830 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f8f78009fb0 tx=0x7f8f78009b30).stop 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 -- 172.21.15.175:0/1793528418 shutdown_connections 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 --2- 172.21.15.175:0/1793528418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c0f5830 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 -- 172.21.15.175:0/1793528418 >> 172.21.15.175:0/1793528418 conn(0x7f8f7c0f0b70 msgr2=0x7f8f7c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 -- 172.21.15.175:0/1793528418 shutdown_connections 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.974+0000 7f8f8333f700 1 -- 172.21.15.175:0/1793528418 wait complete. 2023-12-06T03:24:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f8333f700 1 Processor -- start 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f8333f700 1 -- start start 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f8333f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c106720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f8333f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c106c10 con 0x7f8f7c0f5410 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f810db700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f810db700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33376/0 (socket says 172.21.15.175:33376) 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f810db700 1 -- 172.21.15.175:0/3516778470 learned_addr learned my addr 172.21.15.175:0/3516778470 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f810db700 1 -- 172.21.15.175:0/3516778470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78005040 con 0x7f8f7c0f5410 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.975+0000 7f8f810db700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c106720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f8f78013510 tx=0x7f8f7800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.976+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f78009320 con 0x7f8f7c0f5410 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.976+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f78013f00 con 0x7f8f7c0f5410 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.976+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c106e10 con 0x7f8f7c0f5410 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.976+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f78014d90 con 0x7f8f7c0f5410 2023-12-06T03:24:10.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.976+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c1072a0 con 0x7f8f7c0f5410 2023-12-06T03:24:10.979 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.977+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8f780095f0 con 0x7f8f7c0f5410 2023-12-06T03:24:10.979 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.977+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c045ad0 con 0x7f8f7c0f5410 2023-12-06T03:24:10.979 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.977+0000 7f8f71ffb700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f6802ff60 0x7f8f68032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:10.979 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.977+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8f780410c0 con 0x7f8f7c0f5410 2023-12-06T03:24:10.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.977+0000 7f8f808da700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f6802ff60 0x7f8f68032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:10.980 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.977+0000 7f8f808da700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f6802ff60 0x7f8f68032420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8f6c0096a0 tx=0x7f8f6c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:10.983 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:10.982+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8f78014420 con 0x7f8f7c0f5410 2023-12-06T03:24:11.191 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.189+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8f7c108150 con 0x7f8f7c0f5410 2023-12-06T03:24:11.191 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.190+0000 7f8f71ffb700 1 -- 172.21.15.175:0/3516778470 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8f7801d7f0 con 0x7f8f7c0f5410 2023-12-06T03:24:11.191 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:11.191 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.191+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f6802ff60 msgr2=0x7f8f68032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.191+0000 7f8f8333f700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f6802ff60 0x7f8f68032420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8f6c0096a0 tx=0x7f8f6c006b40).stop 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.191+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 msgr2=0x7f8f7c106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.191+0000 7f8f8333f700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c106720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f8f78013510 tx=0x7f8f7800b040).stop 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.192+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 shutdown_connections 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.192+0000 7f8f8333f700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f6802ff60 0x7f8f68032420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.192+0000 7f8f8333f700 1 --2- 172.21.15.175:0/3516778470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f7c0f5410 0x7f8f7c106720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.192+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 >> 172.21.15.175:0/3516778470 conn(0x7f8f7c0f0b70 msgr2=0x7f8f7c05fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.192+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 shutdown_connections 2023-12-06T03:24:11.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:11.192+0000 7f8f8333f700 1 -- 172.21.15.175:0/3516778470 wait complete. 2023-12-06T03:24:11.194 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:10.429518+0000 mgr.smithi055.qjkkyb (mgr.14164) 85 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:11.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:11.191074+0000 mon.smithi055 (mon.0) 171 : audit [DBG] from='client.? 172.21.15.175:0/3516778470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:12.581 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:12.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:13.756 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 -- 172.21.15.175:0/4008375928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 msgr2=0x7f6ee80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:13.756 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 --2- 172.21.15.175:0/4008375928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80f55c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f6ed8009fb0 tx=0x7f6ed8009b30).stop 2023-12-06T03:24:13.756 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 -- 172.21.15.175:0/4008375928 shutdown_connections 2023-12-06T03:24:13.756 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 --2- 172.21.15.175:0/4008375928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80f55c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:13.756 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 -- 172.21.15.175:0/4008375928 >> 172.21.15.175:0/4008375928 conn(0x7f6ee80f09a0 msgr2=0x7f6ee80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 -- 172.21.15.175:0/4008375928 shutdown_connections 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 -- 172.21.15.175:0/4008375928 wait complete. 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 Processor -- start 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.751+0000 7f6eece7e700 1 -- start start 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6eece7e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6eece7e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80fb5a0 con 0x7f6ee80f51a0 2023-12-06T03:24:13.757 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ee77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33404/0 (socket says 172.21.15.175:33404) 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ee77fe700 1 -- 172.21.15.175:0/2670603723 learned_addr learned my addr 172.21.15.175:0/2670603723 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ee77fe700 1 -- 172.21.15.175:0/2670603723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8005040 con 0x7f6ee80f51a0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ee77fe700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80fd120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6ed8013510 tx=0x7f6ed800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ed8009320 con 0x7f6ee80f51a0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ed8013f00 con 0x7f6ee80f51a0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee80fb740 con 0x7f6ee80f51a0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.752+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ed8014dc0 con 0x7f6ee80f51a0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.753+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee80fbbd0 con 0x7f6ee80f51a0 2023-12-06T03:24:13.758 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.753+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6ed80095f0 con 0x7f6ee80f51a0 2023-12-06T03:24:13.759 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.753+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8059100 con 0x7f6ee80f51a0 2023-12-06T03:24:13.759 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.753+0000 7f6ecffff700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6ed002ff60 0x7f6ed0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:13.759 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.753+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6ed8041100 con 0x7f6ee80f51a0 2023-12-06T03:24:13.759 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.754+0000 7f6ee6ffd700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6ed002ff60 0x7f6ed0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:13.759 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.754+0000 7f6ee6ffd700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6ed002ff60 0x7f6ed0032420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f6edc0096a0 tx=0x7f6edc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:13.760 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.758+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6ed8014420 con 0x7f6ee80f51a0 2023-12-06T03:24:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:12.429817+0000 mgr.smithi055.qjkkyb (mgr.14164) 86 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:13.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.962+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6ee80fabe0 con 0x7f6ee80f51a0 2023-12-06T03:24:13.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.963+0000 7f6ecffff700 1 -- 172.21.15.175:0/2670603723 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6ed8010230 con 0x7f6ee80f51a0 2023-12-06T03:24:13.965 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:13.965 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:13.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6ed002ff60 msgr2=0x7f6ed0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:13.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6ed002ff60 0x7f6ed0032420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f6edc0096a0 tx=0x7f6edc006b40).stop 2023-12-06T03:24:13.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 msgr2=0x7f6ee80fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:13.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80fd120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f6ed8013510 tx=0x7f6ed800b040).stop 2023-12-06T03:24:13.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 shutdown_connections 2023-12-06T03:24:13.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6ed002ff60 0x7f6ed0032420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:13.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 --2- 172.21.15.175:0/2670603723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ee80f51a0 0x7f6ee80fd120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:13.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 >> 172.21.15.175:0/2670603723 conn(0x7f6ee80f09a0 msgr2=0x7f6ee805fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:13.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 shutdown_connections 2023-12-06T03:24:13.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:13.965+0000 7f6eece7e700 1 -- 172.21.15.175:0/2670603723 wait complete. 2023-12-06T03:24:13.967 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:14.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:13.964427+0000 mon.smithi055 (mon.0) 172 : audit [DBG] from='client.? 172.21.15.175:0/2670603723' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:15.390 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:15.390 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:15.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:14.430144+0000 mgr.smithi055.qjkkyb (mgr.14164) 87 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:16.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 -- 172.21.15.175:0/1952253299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 msgr2=0x7f7f5c0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:16.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 --2- 172.21.15.175:0/1952253299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c0f6930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f7f4c009fb0 tx=0x7f7f4c009b30).stop 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 -- 172.21.15.175:0/1952253299 shutdown_connections 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 --2- 172.21.15.175:0/1952253299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c0f6930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 -- 172.21.15.175:0/1952253299 >> 172.21.15.175:0/1952253299 conn(0x7f7f5c0f1c70 msgr2=0x7f7f5c0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 -- 172.21.15.175:0/1952253299 shutdown_connections 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.581+0000 7f7f613df700 1 -- 172.21.15.175:0/1952253299 wait complete. 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.582+0000 7f7f613df700 1 Processor -- start 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.582+0000 7f7f613df700 1 -- start start 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.582+0000 7f7f613df700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:16.584 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.582+0000 7f7f613df700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c106c20 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f5affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f5affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45600/0 (socket says 172.21.15.175:45600) 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f5affd700 1 -- 172.21.15.175:0/1568512553 learned_addr learned my addr 172.21.15.175:0/1568512553 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f5affd700 1 -- 172.21.15.175:0/1568512553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c005040 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f5affd700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c106730 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f7f4c013510 tx=0x7f7f4c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f4c009320 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7f4c013f00 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f4c014d90 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c106e20 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.583+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c1072b0 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.584+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7f4c0095f0 con 0x7f7f5c0f6510 2023-12-06T03:24:16.586 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.584+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c045ad0 con 0x7f7f5c0f6510 2023-12-06T03:24:16.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.584+0000 7f7f437fe700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7f4402ff60 0x7f7f44032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:16.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.584+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7f4c0410c0 con 0x7f7f5c0f6510 2023-12-06T03:24:16.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.584+0000 7f7f5a7fc700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7f4402ff60 0x7f7f44032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:16.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.585+0000 7f7f5a7fc700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7f4402ff60 0x7f7f44032420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f7f500096a0 tx=0x7f7f50006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:16.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.589+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7f4c01d7f0 con 0x7f7f5c0f6510 2023-12-06T03:24:16.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.796+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f5c0fbf60 con 0x7f7f5c0f6510 2023-12-06T03:24:16.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.796+0000 7f7f437fe700 1 -- 172.21.15.175:0/1568512553 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7f4c010230 con 0x7f7f5c0f6510 2023-12-06T03:24:16.798 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:16.798 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:16.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7f4402ff60 msgr2=0x7f7f44032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:16.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7f4402ff60 0x7f7f44032420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f7f500096a0 tx=0x7f7f50006b40).stop 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 msgr2=0x7f7f5c106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c106730 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f7f4c013510 tx=0x7f7f4c00b040).stop 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 shutdown_connections 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7f4402ff60 0x7f7f44032420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 --2- 172.21.15.175:0/1568512553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f5c0f6510 0x7f7f5c106730 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 >> 172.21.15.175:0/1568512553 conn(0x7f7f5c0f1c70 msgr2=0x7f7f5c0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 shutdown_connections 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:16.798+0000 7f7f613df700 1 -- 172.21.15.175:0/1568512553 wait complete. 2023-12-06T03:24:16.800 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:16.430507+0000 mgr.smithi055.qjkkyb (mgr.14164) 88 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:16.797761+0000 mon.smithi055 (mon.0) 173 : audit [DBG] from='client.? 172.21.15.175:0/1568512553' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:18.185 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:18.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.339+0000 7efe73c25700 1 -- 172.21.15.175:0/601243568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 msgr2=0x7efe6c0f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.339+0000 7efe73c25700 1 --2- 172.21.15.175:0/601243568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c0f5d70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7efe68009fb0 tx=0x7efe68009b30).stop 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.339+0000 7efe73c25700 1 -- 172.21.15.175:0/601243568 shutdown_connections 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.339+0000 7efe73c25700 1 --2- 172.21.15.175:0/601243568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c0f5d70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.339+0000 7efe73c25700 1 -- 172.21.15.175:0/601243568 >> 172.21.15.175:0/601243568 conn(0x7efe6c0f0260 msgr2=0x7efe6c0f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.340+0000 7efe73c25700 1 -- 172.21.15.175:0/601243568 shutdown_connections 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.340+0000 7efe73c25700 1 -- 172.21.15.175:0/601243568 wait complete. 2023-12-06T03:24:19.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.340+0000 7efe73c25700 1 Processor -- start 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.340+0000 7efe73c25700 1 -- start start 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.340+0000 7efe73c25700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c105610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.340+0000 7efe73c25700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c105b00 con 0x7efe6c0f5950 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe719c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe719c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45616/0 (socket says 172.21.15.175:45616) 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe719c1700 1 -- 172.21.15.175:0/2969324436 learned_addr learned my addr 172.21.15.175:0/2969324436 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe719c1700 1 -- 172.21.15.175:0/2969324436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68005040 con 0x7efe6c0f5950 2023-12-06T03:24:19.343 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe719c1700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c105610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7efe68013510 tx=0x7efe6800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:19.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe68009320 con 0x7efe6c0f5950 2023-12-06T03:24:19.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe68013f00 con 0x7efe6c0f5950 2023-12-06T03:24:19.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c105d00 con 0x7efe6c0f5950 2023-12-06T03:24:19.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe68014d90 con 0x7efe6c0f5950 2023-12-06T03:24:19.344 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.341+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c106190 con 0x7efe6c0f5950 2023-12-06T03:24:19.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.342+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7efe680095f0 con 0x7efe6c0f5950 2023-12-06T03:24:19.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.342+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c045ad0 con 0x7efe6c0f5950 2023-12-06T03:24:19.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.342+0000 7efe627fc700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe5802ff60 0x7efe58032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:19.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.342+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7efe680410c0 con 0x7efe6c0f5950 2023-12-06T03:24:19.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.342+0000 7efe711c0700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe5802ff60 0x7efe58032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:19.345 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.343+0000 7efe711c0700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe5802ff60 0x7efe58032420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7efe5c0096a0 tx=0x7efe5c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:19.348 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.347+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efe68014420 con 0x7efe6c0f5950 2023-12-06T03:24:19.553 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.552+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efe6c107040 con 0x7efe6c0f5950 2023-12-06T03:24:19.554 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.552+0000 7efe627fc700 1 -- 172.21.15.175:0/2969324436 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efe68010230 con 0x7efe6c0f5950 2023-12-06T03:24:19.554 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:19.554 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe5802ff60 msgr2=0x7efe58032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe5802ff60 0x7efe58032420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7efe5c0096a0 tx=0x7efe5c006b40).stop 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 msgr2=0x7efe6c105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c105610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7efe68013510 tx=0x7efe6800b040).stop 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 shutdown_connections 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe5802ff60 0x7efe58032420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 --2- 172.21.15.175:0/2969324436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe6c0f5950 0x7efe6c105610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 >> 172.21.15.175:0/2969324436 conn(0x7efe6c0f0260 msgr2=0x7efe6c0f0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:19.555 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 shutdown_connections 2023-12-06T03:24:19.556 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:19.554+0000 7efe73c25700 1 -- 172.21.15.175:0/2969324436 wait complete. 2023-12-06T03:24:19.556 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:19.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:18.430757+0000 mgr.smithi055.qjkkyb (mgr.14164) 89 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:20.943 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:20.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:20.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:20 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:19.553665+0000 mon.smithi055 (mon.0) 174 : audit [DBG] from='client.? 172.21.15.175:0/2969324436' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:20.431090+0000 mgr.smithi055.qjkkyb (mgr.14164) 90 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 -- 172.21.15.175:0/1978696622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 msgr2=0x7f943c0f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 --2- 172.21.15.175:0/1978696622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f8e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f942c009fb0 tx=0x7f942c009b30).stop 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 -- 172.21.15.175:0/1978696622 shutdown_connections 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 --2- 172.21.15.175:0/1978696622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f8e20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 -- 172.21.15.175:0/1978696622 >> 172.21.15.175:0/1978696622 conn(0x7f943c0f08c0 msgr2=0x7f943c0f2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 -- 172.21.15.175:0/1978696622 shutdown_connections 2023-12-06T03:24:22.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 -- 172.21.15.175:0/1978696622 wait complete. 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 Processor -- start 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.121+0000 7f94414f5700 1 -- start start 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f94414f5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f94414f5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0f4d20 con 0x7f943c060280 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f943bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f943bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45632/0 (socket says 172.21.15.175:45632) 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f943bfff700 1 -- 172.21.15.175:0/2322123959 learned_addr learned my addr 172.21.15.175:0/2322123959 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f943bfff700 1 -- 172.21.15.175:0/2322123959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c005040 con 0x7f943c060280 2023-12-06T03:24:22.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f943bfff700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f68a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f942c013510 tx=0x7f942c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:22.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f942c009320 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c0f4ec0 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f942c013f00 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f942c014dc0 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.122+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c0f5350 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.123+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f942c0095f0 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.123+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c059100 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.123+0000 7f9438ff9700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f942402ff60 0x7f9424032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.123+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f942c0410e0 con 0x7f943c060280 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.124+0000 7f943b7fe700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f942402ff60 0x7f9424032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:22.126 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.124+0000 7f943b7fe700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f942402ff60 0x7f9424032420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f94300096a0 tx=0x7f9430006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:22.129 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.128+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f942c014420 con 0x7f943c060280 2023-12-06T03:24:22.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.338+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f943c045ad0 con 0x7f943c060280 2023-12-06T03:24:22.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.339+0000 7f9438ff9700 1 -- 172.21.15.175:0/2322123959 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f942c01c920 con 0x7f943c060280 2023-12-06T03:24:22.340 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:22.340 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:22.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f942402ff60 msgr2=0x7f9424032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f942402ff60 0x7f9424032420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f94300096a0 tx=0x7f9430006b40).stop 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 msgr2=0x7f943c0f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f68a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f942c013510 tx=0x7f942c00b040).stop 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 shutdown_connections 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f942402ff60 0x7f9424032420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 --2- 172.21.15.175:0/2322123959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f943c060280 0x7f943c0f68a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 >> 172.21.15.175:0/2322123959 conn(0x7f943c0f08c0 msgr2=0x7f943c0f1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 shutdown_connections 2023-12-06T03:24:22.342 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:22.340+0000 7f94414f5700 1 -- 172.21.15.175:0/2322123959 wait complete. 2023-12-06T03:24:22.343 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:22.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:22.339872+0000 mon.smithi055 (mon.0) 175 : audit [DBG] from='client.? 172.21.15.175:0/2322123959' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:23.715 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:23.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:22.431471+0000 mgr.smithi055.qjkkyb (mgr.14164) 91 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:22.551369+0000 mon.smithi055 (mon.0) 176 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:24:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:22.554862+0000 mon.smithi055 (mon.0) 177 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/3627779678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 msgr2=0x7fe7e00f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 --2- 172.21.15.175:0/3627779678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00f55c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fe7d4009fb0 tx=0x7fe7d4009b30).stop 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/3627779678 shutdown_connections 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 --2- 172.21.15.175:0/3627779678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00f55c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/3627779678 >> 172.21.15.175:0/3627779678 conn(0x7fe7e00f09a0 msgr2=0x7fe7e00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/3627779678 shutdown_connections 2023-12-06T03:24:24.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/3627779678 wait complete. 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 Processor -- start 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.910+0000 7fe7e5f7d700 1 -- start start 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e5f7d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e5f7d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00fb5a0 con 0x7fe7e00f51a0 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e4f7b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e4f7b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45654/0 (socket says 172.21.15.175:45654) 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e4f7b700 1 -- 172.21.15.175:0/837018470 learned_addr learned my addr 172.21.15.175:0/837018470 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e4f7b700 1 -- 172.21.15.175:0/837018470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4005040 con 0x7fe7e00f51a0 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e4f7b700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00fd120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fe7d4013510 tx=0x7fe7d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7d4009320 con 0x7fe7e00f51a0 2023-12-06T03:24:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e00fb740 con 0x7fe7e00f51a0 2023-12-06T03:24:24.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7d4013f00 con 0x7fe7e00f51a0 2023-12-06T03:24:24.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.911+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7d4014dc0 con 0x7fe7e00f51a0 2023-12-06T03:24:24.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.912+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e00fbbd0 con 0x7fe7e00f51a0 2023-12-06T03:24:24.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.912+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fe7d4014420 con 0x7fe7e00f51a0 2023-12-06T03:24:24.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.912+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e0059100 con 0x7fe7e00f51a0 2023-12-06T03:24:24.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.912+0000 7fe7dd7fa700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7d002ff60 0x7fe7d0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:24.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.913+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fe7d40410e0 con 0x7fe7e00f51a0 2023-12-06T03:24:24.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.913+0000 7fe7dffff700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7d002ff60 0x7fe7d0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:24.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.913+0000 7fe7dffff700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7d002ff60 0x7fe7d0032420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe7cc0096a0 tx=0x7fe7cc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:24.919 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:24.917+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe7d40095f0 con 0x7fe7e00f51a0 2023-12-06T03:24:25.122 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.121+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe7e00fabe0 con 0x7fe7e00f51a0 2023-12-06T03:24:25.123 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.121+0000 7fe7dd7fa700 1 -- 172.21.15.175:0/837018470 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe7d401cb10 con 0x7fe7e00f51a0 2023-12-06T03:24:25.123 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:25.123 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:25.124 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7d002ff60 msgr2=0x7fe7d0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7d002ff60 0x7fe7d0032420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe7cc0096a0 tx=0x7fe7cc006b40).stop 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 msgr2=0x7fe7e00fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00fd120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fe7d4013510 tx=0x7fe7d400b040).stop 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 shutdown_connections 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7d002ff60 0x7fe7d0032420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 --2- 172.21.15.175:0/837018470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7e00f51a0 0x7fe7e00fd120 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 >> 172.21.15.175:0/837018470 conn(0x7fe7e00f09a0 msgr2=0x7fe7e005fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 shutdown_connections 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:25.123+0000 7fe7e5f7d700 1 -- 172.21.15.175:0/837018470 wait complete. 2023-12-06T03:24:25.125 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:25.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:24.431758+0000 mgr.smithi055.qjkkyb (mgr.14164) 92 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:26.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:25.122703+0000 mon.smithi055 (mon.0) 178 : audit [DBG] from='client.? 172.21.15.175:0/837018470' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:26.498 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:26.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:27.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:26.432104+0000 mgr.smithi055.qjkkyb (mgr.14164) 93 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 -- 172.21.15.175:0/1304944038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 msgr2=0x7f13800f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 --2- 172.21.15.175:0/1304944038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13800f5830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f1370009fb0 tx=0x7f1370009b30).stop 2023-12-06T03:24:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 -- 172.21.15.175:0/1304944038 shutdown_connections 2023-12-06T03:24:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 --2- 172.21.15.175:0/1304944038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13800f5830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 -- 172.21.15.175:0/1304944038 >> 172.21.15.175:0/1304944038 conn(0x7f13800f0b70 msgr2=0x7f13800f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 -- 172.21.15.175:0/1304944038 shutdown_connections 2023-12-06T03:24:27.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.641+0000 7f13858ad700 1 -- 172.21.15.175:0/1304944038 wait complete. 2023-12-06T03:24:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f13858ad700 1 Processor -- start 2023-12-06T03:24:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f13858ad700 1 -- start start 2023-12-06T03:24:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f13858ad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13801093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f13858ad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1380060280 con 0x7f13800f5410 2023-12-06T03:24:27.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f137effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13801093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f137effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13801093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57082/0 (socket says 172.21.15.175:57082) 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.642+0000 7f137effd700 1 -- 172.21.15.175:0/4093398792 learned_addr learned my addr 172.21.15.175:0/4093398792 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f137effd700 1 -- 172.21.15.175:0/4093398792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1370005040 con 0x7f13800f5410 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f137effd700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13801093f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f1370013510 tx=0x7f137000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1370009320 con 0x7f13800f5410 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1380060480 con 0x7f13800f5410 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1370013f00 con 0x7f13800f5410 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1370014d90 con 0x7f13800f5410 2023-12-06T03:24:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.643+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1380109b20 con 0x7f13800f5410 2023-12-06T03:24:27.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.644+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f13700095f0 con 0x7f13800f5410 2023-12-06T03:24:27.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.644+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1380045ad0 con 0x7f13800f5410 2023-12-06T03:24:27.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.644+0000 7f13777fe700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f136c02ff60 0x7f136c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:27.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.644+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f1370041000 con 0x7f13800f5410 2023-12-06T03:24:27.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.644+0000 7f137e7fc700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f136c02ff60 0x7f136c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:27.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.644+0000 7f137e7fc700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f136c02ff60 0x7f136c032420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f13680096a0 tx=0x7f1368006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:27.649 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.649+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1370014420 con 0x7f13800f5410 2023-12-06T03:24:27.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.855+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f138010a340 con 0x7f13800f5410 2023-12-06T03:24:27.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.856+0000 7f13777fe700 1 -- 172.21.15.175:0/4093398792 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f1370010230 con 0x7f13800f5410 2023-12-06T03:24:27.856 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:27.856 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f136c02ff60 msgr2=0x7f136c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f136c02ff60 0x7f136c032420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f13680096a0 tx=0x7f1368006b40).stop 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 msgr2=0x7f13801093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13801093f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f1370013510 tx=0x7f137000b040).stop 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 shutdown_connections 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f136c02ff60 0x7f136c032420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 --2- 172.21.15.175:0/4093398792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13800f5410 0x7f13801093f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.857+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 >> 172.21.15.175:0/4093398792 conn(0x7f13800f0b70 msgr2=0x7f13800f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.858+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 shutdown_connections 2023-12-06T03:24:27.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:27.858+0000 7f13858ad700 1 -- 172.21.15.175:0/4093398792 wait complete. 2023-12-06T03:24:27.859 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:28.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:27.855996+0000 mon.smithi055 (mon.0) 179 : audit [DBG] from='client.? 172.21.15.175:0/4093398792' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:29.237 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:29.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:29.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:28.432497+0000 mgr.smithi055.qjkkyb (mgr.14164) 94 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 -- 172.21.15.175:0/1870487878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 msgr2=0x7fd3140f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 --2- 172.21.15.175:0/1870487878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3140f66c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fd310009fb0 tx=0x7fd310009b30).stop 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 -- 172.21.15.175:0/1870487878 shutdown_connections 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 --2- 172.21.15.175:0/1870487878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3140f66c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 -- 172.21.15.175:0/1870487878 >> 172.21.15.175:0/1870487878 conn(0x7fd3140f1aa0 msgr2=0x7fd3140f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 -- 172.21.15.175:0/1870487878 shutdown_connections 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.438+0000 7fd31abcd700 1 -- 172.21.15.175:0/1870487878 wait complete. 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.439+0000 7fd31abcd700 1 Processor -- start 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.439+0000 7fd31abcd700 1 -- start start 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.439+0000 7fd31abcd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3141064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:30.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.439+0000 7fd31abcd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141069a0 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd319bcb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3141064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd319bcb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3141064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57100/0 (socket says 172.21.15.175:57100) 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd319bcb700 1 -- 172.21.15.175:0/2495463586 learned_addr learned my addr 172.21.15.175:0/2495463586 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd319bcb700 1 -- 172.21.15.175:0/2495463586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd310005040 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd319bcb700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3141064b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fd310013510 tx=0x7fd31000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd310009320 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd314106ba0 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd310013f00 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd310014dc0 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.440+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd314107030 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.441+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd31001d3d0 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.441+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd314045ad0 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.441+0000 7fd30a7fc700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd30002ff60 0x7fd300032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.441+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd3100410f0 con 0x7fd3140f62a0 2023-12-06T03:24:30.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.442+0000 7fd3193ca700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd30002ff60 0x7fd300032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:30.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.442+0000 7fd3193ca700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd30002ff60 0x7fd300032420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd3040096a0 tx=0x7fd304006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:30.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.447+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd3100126f0 con 0x7fd3140f62a0 2023-12-06T03:24:30.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.651+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd314107ee0 con 0x7fd3140f62a0 2023-12-06T03:24:30.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.652+0000 7fd30a7fc700 1 -- 172.21.15.175:0/2495463586 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd310012330 con 0x7fd3140f62a0 2023-12-06T03:24:30.653 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:30.653 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:30.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd30002ff60 msgr2=0x7fd300032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd30002ff60 0x7fd300032420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd3040096a0 tx=0x7fd304006b40).stop 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 msgr2=0x7fd3141064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3141064b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fd310013510 tx=0x7fd31000b040).stop 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 shutdown_connections 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd30002ff60 0x7fd300032420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 --2- 172.21.15.175:0/2495463586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd3140f62a0 0x7fd3141064b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 >> 172.21.15.175:0/2495463586 conn(0x7fd3140f1aa0 msgr2=0x7fd3140f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 shutdown_connections 2023-12-06T03:24:30.655 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:30.653+0000 7fd31abcd700 1 -- 172.21.15.175:0/2495463586 wait complete. 2023-12-06T03:24:30.656 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:30.432850+0000 mgr.smithi055.qjkkyb (mgr.14164) 95 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:30.652778+0000 mon.smithi055 (mon.0) 180 : audit [DBG] from='client.? 172.21.15.175:0/2495463586' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:32.078 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:32.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.248+0000 7f0b289a1700 1 -- 172.21.15.175:0/3967141816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 msgr2=0x7f0b240f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.248+0000 7f0b289a1700 1 --2- 172.21.15.175:0/3967141816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b240f55c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f0b18009fb0 tx=0x7f0b18009b30).stop 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.248+0000 7f0b289a1700 1 -- 172.21.15.175:0/3967141816 shutdown_connections 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.248+0000 7f0b289a1700 1 --2- 172.21.15.175:0/3967141816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b240f55c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.248+0000 7f0b289a1700 1 -- 172.21.15.175:0/3967141816 >> 172.21.15.175:0/3967141816 conn(0x7f0b240f09a0 msgr2=0x7f0b240f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.249+0000 7f0b289a1700 1 -- 172.21.15.175:0/3967141816 shutdown_connections 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.249+0000 7f0b289a1700 1 -- 172.21.15.175:0/3967141816 wait complete. 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.249+0000 7f0b289a1700 1 Processor -- start 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.249+0000 7f0b289a1700 1 -- start start 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.249+0000 7f0b289a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b24109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:33.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.249+0000 7f0b289a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b24060280 con 0x7f0b240f51a0 2023-12-06T03:24:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b237fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b24109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b237fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b24109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57122/0 (socket says 172.21.15.175:57122) 2023-12-06T03:24:33.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b237fe700 1 -- 172.21.15.175:0/1729268202 learned_addr learned my addr 172.21.15.175:0/1729268202 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b237fe700 1 -- 172.21.15.175:0/1729268202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18005040 con 0x7f0b240f51a0 2023-12-06T03:24:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b237fe700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b24109170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f0b18013510 tx=0x7f0b1800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b18009320 con 0x7f0b240f51a0 2023-12-06T03:24:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b24060480 con 0x7f0b240f51a0 2023-12-06T03:24:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b18013f00 con 0x7f0b240f51a0 2023-12-06T03:24:33.253 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b18014dc0 con 0x7f0b240f51a0 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.250+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b241098a0 con 0x7f0b240f51a0 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.251+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0b18014420 con 0x7f0b240f51a0 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.251+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b24045ad0 con 0x7f0b240f51a0 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.251+0000 7f0b0bfff700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b0c02ff60 0x7f0b0c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.252+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0b18041050 con 0x7f0b240f51a0 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.252+0000 7f0b22ffd700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b0c02ff60 0x7f0b0c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:33.254 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.252+0000 7f0b22ffd700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b0c02ff60 0x7f0b0c032420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f0b140096a0 tx=0x7f0b14006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:33.257 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.256+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0b180095f0 con 0x7f0b240f51a0 2023-12-06T03:24:33.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.464+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b240fabe0 con 0x7f0b240f51a0 2023-12-06T03:24:33.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.464+0000 7f0b0bfff700 1 -- 172.21.15.175:0/1729268202 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0b1801cb10 con 0x7f0b240f51a0 2023-12-06T03:24:33.466 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:33.466 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:33.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b0c02ff60 msgr2=0x7f0b0c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:33.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b0c02ff60 0x7f0b0c032420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f0b140096a0 tx=0x7f0b14006b40).stop 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 msgr2=0x7f0b24109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b24109170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f0b18013510 tx=0x7f0b1800b040).stop 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 shutdown_connections 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b0c02ff60 0x7f0b0c032420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 --2- 172.21.15.175:0/1729268202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b240f51a0 0x7f0b24109170 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 >> 172.21.15.175:0/1729268202 conn(0x7f0b240f09a0 msgr2=0x7f0b240f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 shutdown_connections 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:33.466+0000 7f0b289a1700 1 -- 172.21.15.175:0/1729268202 wait complete. 2023-12-06T03:24:33.468 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:32.433182+0000 mgr.smithi055.qjkkyb (mgr.14164) 96 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:33.465621+0000 mon.smithi055 (mon.0) 181 : audit [DBG] from='client.? 172.21.15.175:0/1729268202' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:34.868 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:34.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:34.433538+0000 mgr.smithi055.qjkkyb (mgr.14164) 97 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.022+0000 7f97ae4d0700 1 -- 172.21.15.175:0/1629834634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 msgr2=0x7f97a80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.022+0000 7f97ae4d0700 1 --2- 172.21.15.175:0/1629834634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a80f55c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9798009fb0 tx=0x7f9798009b30).stop 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.022+0000 7f97ae4d0700 1 -- 172.21.15.175:0/1629834634 shutdown_connections 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.022+0000 7f97ae4d0700 1 --2- 172.21.15.175:0/1629834634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a80f55c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.022+0000 7f97ae4d0700 1 -- 172.21.15.175:0/1629834634 >> 172.21.15.175:0/1629834634 conn(0x7f97a80f09a0 msgr2=0x7f97a80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.023+0000 7f97ae4d0700 1 -- 172.21.15.175:0/1629834634 shutdown_connections 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.023+0000 7f97ae4d0700 1 -- 172.21.15.175:0/1629834634 wait complete. 2023-12-06T03:24:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.023+0000 7f97ae4d0700 1 Processor -- start 2023-12-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.023+0000 7f97ae4d0700 1 -- start start 2023-12-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.023+0000 7f97ae4d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a8109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.023+0000 7f97ae4d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a8060280 con 0x7f97a80f51a0 2023-12-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f97ad4ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a8109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f97ad4ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a8109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:54716/0 (socket says 172.21.15.175:54716) 2023-12-06T03:24:36.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f97ad4ce700 1 -- 172.21.15.175:0/2656401660 learned_addr learned my addr 172.21.15.175:0/2656401660 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:36.027 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f97ad4ce700 1 -- 172.21.15.175:0/2656401660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798005040 con 0x7f97a80f51a0 2023-12-06T03:24:36.027 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f97ad4ce700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a8109170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f9798013510 tx=0x7f979800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9798009320 con 0x7f97a80f51a0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a8060480 con 0x7f97a80f51a0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9798013f00 con 0x7f97a80f51a0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.024+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9798014dc0 con 0x7f97a80f51a0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.025+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a81098a0 con 0x7f97a80f51a0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.025+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a8045ad0 con 0x7f97a80f51a0 2023-12-06T03:24:36.028 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.025+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f97980095f0 con 0x7f97a80f51a0 2023-12-06T03:24:36.029 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.026+0000 7f979dffb700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f979402ff60 0x7f9794032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:36.029 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.026+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9798041020 con 0x7f97a80f51a0 2023-12-06T03:24:36.029 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.026+0000 7f97acccd700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f979402ff60 0x7f9794032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:36.029 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.026+0000 7f97acccd700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f979402ff60 0x7f9794032420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f97a40096a0 tx=0x7f97a4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:36.032 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.031+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9798014420 con 0x7f97a80f51a0 2023-12-06T03:24:36.242 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.240+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f97a80fabe0 con 0x7f97a80f51a0 2023-12-06T03:24:36.242 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.241+0000 7f979dffb700 1 -- 172.21.15.175:0/2656401660 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f979801c920 con 0x7f97a80f51a0 2023-12-06T03:24:36.243 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:36.243 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:36.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f979402ff60 msgr2=0x7f9794032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:36.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f979402ff60 0x7f9794032420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f97a40096a0 tx=0x7f97a4006b40).stop 2023-12-06T03:24:36.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 msgr2=0x7f97a8109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a8109170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f9798013510 tx=0x7f979800b040).stop 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 shutdown_connections 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f979402ff60 0x7f9794032420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 --2- 172.21.15.175:0/2656401660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97a80f51a0 0x7f97a8109170 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 >> 172.21.15.175:0/2656401660 conn(0x7f97a80f09a0 msgr2=0x7f97a80f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 shutdown_connections 2023-12-06T03:24:36.245 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:36.243+0000 7f97ae4d0700 1 -- 172.21.15.175:0/2656401660 wait complete. 2023-12-06T03:24:36.246 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:36.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:36.242389+0000 mon.smithi055 (mon.0) 182 : audit [DBG] from='client.? 172.21.15.175:0/2656401660' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:37.631 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:37.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:37.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:36.433879+0000 mgr.smithi055.qjkkyb (mgr.14164) 98 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:38.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 -- 172.21.15.175:0/2595464344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 msgr2=0x7f652c0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:38.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 --2- 172.21.15.175:0/2595464344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c0f90f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f6528009fb0 tx=0x7f6528009b30).stop 2023-12-06T03:24:38.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 -- 172.21.15.175:0/2595464344 shutdown_connections 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 --2- 172.21.15.175:0/2595464344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c0f90f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 -- 172.21.15.175:0/2595464344 >> 172.21.15.175:0/2595464344 conn(0x7f652c0f0b70 msgr2=0x7f652c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 -- 172.21.15.175:0/2595464344 shutdown_connections 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.858+0000 7f6533bd5700 1 -- 172.21.15.175:0/2595464344 wait complete. 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6533bd5700 1 Processor -- start 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6533bd5700 1 -- start start 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6533bd5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c1066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6533bd5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c106bd0 con 0x7f652c060280 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6531971700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c1066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6531971700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c1066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:54732/0 (socket says 172.21.15.175:54732) 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.859+0000 7f6531971700 1 -- 172.21.15.175:0/4018842279 learned_addr learned my addr 172.21.15.175:0/4018842279 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f6531971700 1 -- 172.21.15.175:0/4018842279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528005040 con 0x7f652c060280 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f6531971700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c1066e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f6528009830 tx=0x7f652800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6528009320 con 0x7f652c060280 2023-12-06T03:24:38.862 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c106dd0 con 0x7f652c060280 2023-12-06T03:24:38.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65280136c0 con 0x7f652c060280 2023-12-06T03:24:38.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f652801ee90 con 0x7f652c060280 2023-12-06T03:24:38.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.860+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c107260 con 0x7f652c060280 2023-12-06T03:24:38.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.861+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f652801e420 con 0x7f652c060280 2023-12-06T03:24:38.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.861+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c045ad0 con 0x7f652c060280 2023-12-06T03:24:38.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.861+0000 7f65227fc700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f651802ff10 0x7f65180323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:38.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.861+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6528041010 con 0x7f652c060280 2023-12-06T03:24:38.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.861+0000 7f6531170700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f651802ff10 0x7f65180323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:38.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.861+0000 7f6531170700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f651802ff10 0x7f65180323d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f651c0096a0 tx=0x7f651c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:38.867 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:38.866+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6528014440 con 0x7f652c060280 2023-12-06T03:24:39.071 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.070+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f652c108110 con 0x7f652c060280 2023-12-06T03:24:39.071 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.070+0000 7f65227fc700 1 -- 172.21.15.175:0/4018842279 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f652801cb10 con 0x7f652c060280 2023-12-06T03:24:39.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:39.072 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:39.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f651802ff10 msgr2=0x7f65180323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:39.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f651802ff10 0x7f65180323d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f651c0096a0 tx=0x7f651c006b40).stop 2023-12-06T03:24:39.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 msgr2=0x7f652c1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:39.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c1066e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f6528009830 tx=0x7f652800b040).stop 2023-12-06T03:24:39.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 shutdown_connections 2023-12-06T03:24:39.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f651802ff10 0x7f65180323d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:39.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 --2- 172.21.15.175:0/4018842279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f652c060280 0x7f652c1066e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:39.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 >> 172.21.15.175:0/4018842279 conn(0x7f652c0f0b70 msgr2=0x7f652c0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:39.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 shutdown_connections 2023-12-06T03:24:39.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:39.072+0000 7f6533bd5700 1 -- 172.21.15.175:0/4018842279 wait complete. 2023-12-06T03:24:39.075 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:38.434185+0000 mgr.smithi055.qjkkyb (mgr.14164) 99 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:39.071662+0000 mon.smithi055 (mon.0) 183 : audit [DBG] from='client.? 172.21.15.175:0/4018842279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:40.476 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:40.476 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:41.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 -- 172.21.15.175:0/2106856352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 msgr2=0x7f61480f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:41.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 --2- 172.21.15.175:0/2106856352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f61480f6930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f613c009fb0 tx=0x7f613c009b30).stop 2023-12-06T03:24:41.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 -- 172.21.15.175:0/2106856352 shutdown_connections 2023-12-06T03:24:41.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 --2- 172.21.15.175:0/2106856352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f61480f6930 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:41.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 -- 172.21.15.175:0/2106856352 >> 172.21.15.175:0/2106856352 conn(0x7f61480f1c70 msgr2=0x7f61480f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:41.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 -- 172.21.15.175:0/2106856352 shutdown_connections 2023-12-06T03:24:41.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.658+0000 7f614ede6700 1 -- 172.21.15.175:0/2106856352 wait complete. 2023-12-06T03:24:41.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614ede6700 1 Processor -- start 2023-12-06T03:24:41.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614ede6700 1 -- start start 2023-12-06T03:24:41.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614ede6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f6148106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:41.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614ede6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148106c20 con 0x7f61480f6510 2023-12-06T03:24:41.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614cb82700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f6148106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614cb82700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f6148106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:54746/0 (socket says 172.21.15.175:54746) 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.659+0000 7f614cb82700 1 -- 172.21.15.175:0/4255027350 learned_addr learned my addr 172.21.15.175:0/4255027350 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f614cb82700 1 -- 172.21.15.175:0/4255027350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c005040 con 0x7f61480f6510 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f614cb82700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f6148106730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f613c013510 tx=0x7f613c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f613c009320 con 0x7f61480f6510 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6148106e20 con 0x7f61480f6510 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f613c013f00 con 0x7f61480f6510 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f613c014d90 con 0x7f61480f6510 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.660+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61481072b0 con 0x7f61480f6510 2023-12-06T03:24:41.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.661+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f613c0095f0 con 0x7f61480f6510 2023-12-06T03:24:41.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.661+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6148045ad0 con 0x7f61480f6510 2023-12-06T03:24:41.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.661+0000 7f61457fa700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f613802ff60 0x7f6138032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:41.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.661+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f613c040fd0 con 0x7f61480f6510 2023-12-06T03:24:41.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.661+0000 7f6147fff700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f613802ff60 0x7f6138032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:41.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.662+0000 7f6147fff700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f613802ff60 0x7f6138032420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f61340096a0 tx=0x7f6134006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:41.667 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.666+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f613c014420 con 0x7f61480f6510 2023-12-06T03:24:41.872 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.871+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f61480fbf60 con 0x7f61480f6510 2023-12-06T03:24:41.873 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.872+0000 7f61457fa700 1 -- 172.21.15.175:0/4255027350 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f613c01d7f0 con 0x7f61480f6510 2023-12-06T03:24:41.873 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:41.873 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.873+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f613802ff60 msgr2=0x7f6138032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.873+0000 7f614ede6700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f613802ff60 0x7f6138032420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f61340096a0 tx=0x7f6134006b40).stop 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.873+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 msgr2=0x7f6148106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.873+0000 7f614ede6700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f6148106730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f613c013510 tx=0x7f613c00b040).stop 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.873+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 shutdown_connections 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.874+0000 7f614ede6700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f613802ff60 0x7f6138032420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.874+0000 7f614ede6700 1 --2- 172.21.15.175:0/4255027350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61480f6510 0x7f6148106730 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.874+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 >> 172.21.15.175:0/4255027350 conn(0x7f61480f1c70 msgr2=0x7f61480f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.874+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 shutdown_connections 2023-12-06T03:24:41.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:41.874+0000 7f614ede6700 1 -- 172.21.15.175:0/4255027350 wait complete. 2023-12-06T03:24:41.876 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:40.434422+0000 mgr.smithi055.qjkkyb (mgr.14164) 100 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:42.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:42 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:41.873010+0000 mon.smithi055 (mon.0) 184 : audit [DBG] from='client.? 172.21.15.175:0/4255027350' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:43.267 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:43.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:42.434756+0000 mgr.smithi055.qjkkyb (mgr.14164) 101 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 -- 172.21.15.175:0/4255109138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 msgr2=0x7f9d940f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 --2- 172.21.15.175:0/4255109138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d940f5830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f9d88009fb0 tx=0x7f9d88009b30).stop 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 -- 172.21.15.175:0/4255109138 shutdown_connections 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 --2- 172.21.15.175:0/4255109138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d940f5830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 -- 172.21.15.175:0/4255109138 >> 172.21.15.175:0/4255109138 conn(0x7f9d940f0b70 msgr2=0x7f9d940f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 -- 172.21.15.175:0/4255109138 shutdown_connections 2023-12-06T03:24:44.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.392+0000 7f9d9b972700 1 -- 172.21.15.175:0/4255109138 wait complete. 2023-12-06T03:24:44.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9b972700 1 Processor -- start 2023-12-06T03:24:44.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9b972700 1 -- start start 2023-12-06T03:24:44.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9b972700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d94109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:44.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9b972700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94060220 con 0x7f9d940f5410 2023-12-06T03:24:44.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9970e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d94109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:44.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9970e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d94109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:54770/0 (socket says 172.21.15.175:54770) 2023-12-06T03:24:44.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.393+0000 7f9d9970e700 1 -- 172.21.15.175:0/3993117576 learned_addr learned my addr 172.21.15.175:0/3993117576 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:44.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d9970e700 1 -- 172.21.15.175:0/3993117576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88005040 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d9970e700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d94109450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f9d88013510 tx=0x7f9d8800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d88009320 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d94060420 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d88013f00 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d88014d90 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.394+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d94109b00 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.395+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9d88014420 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.395+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d94059100 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.395+0000 7f9d85ffb700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9d8002ff60 0x7f9d80032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.395+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9d88041110 con 0x7f9d940f5410 2023-12-06T03:24:44.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.395+0000 7f9d98f0d700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9d8002ff60 0x7f9d80032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:44.398 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.396+0000 7f9d98f0d700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9d8002ff60 0x7f9d80032420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f9d900096a0 tx=0x7f9d90006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:44.401 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.400+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9d880095f0 con 0x7f9d940f5410 2023-12-06T03:24:44.607 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.606+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9d94045ad0 con 0x7f9d940f5410 2023-12-06T03:24:44.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.607+0000 7f9d85ffb700 1 -- 172.21.15.175:0/3993117576 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9d8801db70 con 0x7f9d940f5410 2023-12-06T03:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9d8002ff60 msgr2=0x7f9d80032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9d8002ff60 0x7f9d80032420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f9d900096a0 tx=0x7f9d90006b40).stop 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 msgr2=0x7f9d94109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d94109450 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f9d88013510 tx=0x7f9d8800b040).stop 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 shutdown_connections 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9d8002ff60 0x7f9d80032420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 --2- 172.21.15.175:0/3993117576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d940f5410 0x7f9d94109450 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 >> 172.21.15.175:0/3993117576 conn(0x7f9d940f0b70 msgr2=0x7f9d940f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 shutdown_connections 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:44.609+0000 7f9d9b972700 1 -- 172.21.15.175:0/3993117576 wait complete. 2023-12-06T03:24:44.611 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:44.435033+0000 mgr.smithi055.qjkkyb (mgr.14164) 102 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:45.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:44.608588+0000 mon.smithi055 (mon.0) 185 : audit [DBG] from='client.? 172.21.15.175:0/3993117576' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:45.985 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:45.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- 172.21.15.175:0/4271240230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 msgr2=0x7feac80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 --2- 172.21.15.175:0/4271240230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac80f5830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7feabc009fb0 tx=0x7feabc009b30).stop 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- 172.21.15.175:0/4271240230 shutdown_connections 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 --2- 172.21.15.175:0/4271240230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac80f5830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- 172.21.15.175:0/4271240230 >> 172.21.15.175:0/4271240230 conn(0x7feac80f0b70 msgr2=0x7feac80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- 172.21.15.175:0/4271240230 shutdown_connections 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- 172.21.15.175:0/4271240230 wait complete. 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 Processor -- start 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- start start 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac81066d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.161+0000 7feacff71700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8106bc0 con 0x7feac80f5410 2023-12-06T03:24:47.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacdd0d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac81066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacdd0d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac81066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35416/0 (socket says 172.21.15.175:35416) 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacdd0d700 1 -- 172.21.15.175:0/1825101875 learned_addr learned my addr 172.21.15.175:0/1825101875 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacdd0d700 1 -- 172.21.15.175:0/1825101875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc005040 con 0x7feac80f5410 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacdd0d700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac81066d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7feabc013510 tx=0x7feabc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feabc009320 con 0x7feac80f5410 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac8106dc0 con 0x7feac80f5410 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feabc013f00 con 0x7feac80f5410 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feabc014d90 con 0x7feac80f5410 2023-12-06T03:24:47.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.162+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac8107250 con 0x7feac80f5410 2023-12-06T03:24:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.163+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac8045ad0 con 0x7feac80f5410 2023-12-06T03:24:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.163+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7feabc0095f0 con 0x7feac80f5410 2023-12-06T03:24:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.163+0000 7feaba7fc700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feab402ff60 0x7feab4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.164+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7feabc040fe0 con 0x7feac80f5410 2023-12-06T03:24:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.164+0000 7feacd50c700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feab402ff60 0x7feab4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:47.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.164+0000 7feacd50c700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feab402ff60 0x7feab4032420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7feac40096a0 tx=0x7feac4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:47.169 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.168+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7feabc01d7f0 con 0x7feac80f5410 2023-12-06T03:24:47.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.373+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feac80fae60 con 0x7feac80f5410 2023-12-06T03:24:47.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.374+0000 7feaba7fc700 1 -- 172.21.15.175:0/1825101875 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7feabc010230 con 0x7feac80f5410 2023-12-06T03:24:47.376 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:47.376 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:47.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feab402ff60 msgr2=0x7feab4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:47.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feab402ff60 0x7feab4032420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7feac40096a0 tx=0x7feac4006b40).stop 2023-12-06T03:24:47.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 msgr2=0x7feac81066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac81066d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7feabc013510 tx=0x7feabc00b040).stop 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 shutdown_connections 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feab402ff60 0x7feab4032420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 --2- 172.21.15.175:0/1825101875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feac80f5410 0x7feac81066d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 >> 172.21.15.175:0/1825101875 conn(0x7feac80f0b70 msgr2=0x7feac805fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 shutdown_connections 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:47.376+0000 7feacff71700 1 -- 172.21.15.175:0/1825101875 wait complete. 2023-12-06T03:24:47.378 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:46.435341+0000 mgr.smithi055.qjkkyb (mgr.14164) 103 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:47.375352+0000 mon.smithi055 (mon.0) 186 : audit [DBG] from='client.? 172.21.15.175:0/1825101875' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:48.775 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:48.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:48.435606+0000 mgr.smithi055.qjkkyb (mgr.14164) 104 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 -- 172.21.15.175:0/1628506154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 msgr2=0x7fc8c40f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 --2- 172.21.15.175:0/1628506154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40f55c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fc8b4009fb0 tx=0x7fc8b4009b30).stop 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 -- 172.21.15.175:0/1628506154 shutdown_connections 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 --2- 172.21.15.175:0/1628506154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40f55c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 -- 172.21.15.175:0/1628506154 >> 172.21.15.175:0/1628506154 conn(0x7fc8c40f09a0 msgr2=0x7fc8c40f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 -- 172.21.15.175:0/1628506154 shutdown_connections 2023-12-06T03:24:49.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.954+0000 7fc8ca85b700 1 -- 172.21.15.175:0/1628506154 wait complete. 2023-12-06T03:24:49.958 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8ca85b700 1 Processor -- start 2023-12-06T03:24:49.958 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8ca85b700 1 -- start start 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8ca85b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8ca85b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40f9830 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8c9859700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8c9859700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40fb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35442/0 (socket says 172.21.15.175:35442) 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.955+0000 7fc8c9859700 1 -- 172.21.15.175:0/3985052263 learned_addr learned my addr 172.21.15.175:0/3985052263 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8c9859700 1 -- 172.21.15.175:0/3985052263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4005040 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8c9859700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40fb3b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fc8b4013510 tx=0x7fc8b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8b4009320 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8b4013f00 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c40f99d0 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8b4014dc0 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.956+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c40f9e60 con 0x7fc8c40f51a0 2023-12-06T03:24:49.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.957+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fc8b40095f0 con 0x7fc8c40f51a0 2023-12-06T03:24:49.960 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.957+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c4059100 con 0x7fc8c40f51a0 2023-12-06T03:24:49.960 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.957+0000 7fc8ba7fc700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc8b002ff60 0x7fc8b0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:49.960 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.957+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fc8b40410c0 con 0x7fc8c40f51a0 2023-12-06T03:24:49.960 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.957+0000 7fc8c9058700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc8b002ff60 0x7fc8b0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:49.960 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.957+0000 7fc8c9058700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc8b002ff60 0x7fc8b0032420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc8c00096a0 tx=0x7fc8c0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:49.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:49.962+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc8b401c790 con 0x7fc8c40f51a0 2023-12-06T03:24:50.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.171+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc8c40fbce0 con 0x7fc8c40f51a0 2023-12-06T03:24:50.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.172+0000 7fc8ba7fc700 1 -- 172.21.15.175:0/3985052263 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc8b4010230 con 0x7fc8c40f51a0 2023-12-06T03:24:50.173 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:50.173 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:50.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc8b002ff60 msgr2=0x7fc8b0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:50.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc8b002ff60 0x7fc8b0032420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc8c00096a0 tx=0x7fc8c0006b40).stop 2023-12-06T03:24:50.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 msgr2=0x7fc8c40fb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:50.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40fb3b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fc8b4013510 tx=0x7fc8b400b040).stop 2023-12-06T03:24:50.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 shutdown_connections 2023-12-06T03:24:50.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc8b002ff60 0x7fc8b0032420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:50.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 --2- 172.21.15.175:0/3985052263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8c40f51a0 0x7fc8c40fb3b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:50.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 >> 172.21.15.175:0/3985052263 conn(0x7fc8c40f09a0 msgr2=0x7fc8c40f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:50.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 shutdown_connections 2023-12-06T03:24:50.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:50.174+0000 7fc8ca85b700 1 -- 172.21.15.175:0/3985052263 wait complete. 2023-12-06T03:24:50.176 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:50.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:50 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:50.173043+0000 mon.smithi055 (mon.0) 187 : audit [DBG] from='client.? 172.21.15.175:0/3985052263' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:51.567 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:51.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:50.435893+0000 mgr.smithi055.qjkkyb (mgr.14164) 105 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:52.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- 172.21.15.175:0/3699286816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 msgr2=0x7efeac0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 --2- 172.21.15.175:0/3699286816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0f55c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7efe9c009fb0 tx=0x7efe9c009b30).stop 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- 172.21.15.175:0/3699286816 shutdown_connections 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 --2- 172.21.15.175:0/3699286816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0f55c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- 172.21.15.175:0/3699286816 >> 172.21.15.175:0/3699286816 conn(0x7efeac0f09a0 msgr2=0x7efeac0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- 172.21.15.175:0/3699286816 shutdown_connections 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- 172.21.15.175:0/3699286816 wait complete. 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 Processor -- start 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- start start 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.724+0000 7efeb27a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0fb5a0 con 0x7efeac0f51a0 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efeb179f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efeb179f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35450/0 (socket says 172.21.15.175:35450) 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efeb179f700 1 -- 172.21.15.175:0/3163452396 learned_addr learned my addr 172.21.15.175:0/3163452396 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efeb179f700 1 -- 172.21.15.175:0/3163452396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c005040 con 0x7efeac0f51a0 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efeb179f700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0fd120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7efe9c009830 tx=0x7efe9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe9c009320 con 0x7efeac0f51a0 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac0fb740 con 0x7efeac0f51a0 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efe9c0136c0 con 0x7efeac0f51a0 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.725+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe9c01ee90 con 0x7efeac0f51a0 2023-12-06T03:24:52.728 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.726+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac0fbbd0 con 0x7efeac0f51a0 2023-12-06T03:24:52.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.726+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7efe9c01e420 con 0x7efeac0f51a0 2023-12-06T03:24:52.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.726+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac059100 con 0x7efeac0f51a0 2023-12-06T03:24:52.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.726+0000 7efea1ffb700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe9802ff10 0x7efe980323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:52.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.726+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7efe9c040f40 con 0x7efeac0f51a0 2023-12-06T03:24:52.729 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.727+0000 7efeb0f9e700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe9802ff10 0x7efe980323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:52.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.727+0000 7efeb0f9e700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe9802ff10 0x7efe980323d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7efea80096a0 tx=0x7efea8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:52.732 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.731+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efe9c06d2b0 con 0x7efeac0f51a0 2023-12-06T03:24:52.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.939+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efeac045ad0 con 0x7efeac0f51a0 2023-12-06T03:24:52.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.940+0000 7efea1ffb700 1 -- 172.21.15.175:0/3163452396 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efe9c0095f0 con 0x7efeac0f51a0 2023-12-06T03:24:52.941 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:52.942 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe9802ff10 msgr2=0x7efe980323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe9802ff10 0x7efe980323d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7efea80096a0 tx=0x7efea8006b40).stop 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 msgr2=0x7efeac0fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0fd120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7efe9c009830 tx=0x7efe9c00b040).stop 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 shutdown_connections 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efe9802ff10 0x7efe980323d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:52.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 --2- 172.21.15.175:0/3163452396 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efeac0f51a0 0x7efeac0fd120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:52.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 >> 172.21.15.175:0/3163452396 conn(0x7efeac0f09a0 msgr2=0x7efeac05fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:52.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 shutdown_connections 2023-12-06T03:24:52.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:52.942+0000 7efeb27a1700 1 -- 172.21.15.175:0/3163452396 wait complete. 2023-12-06T03:24:52.944 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:52.436167+0000 mgr.smithi055.qjkkyb (mgr.14164) 106 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:52.941189+0000 mon.smithi055 (mon.0) 188 : audit [DBG] from='client.? 172.21.15.175:0/3163452396' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:54.323 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:54.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:54.436514+0000 mgr.smithi055.qjkkyb (mgr.14164) 107 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 -- 172.21.15.175:0/645624622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 msgr2=0x7f85540f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 --2- 172.21.15.175:0/645624622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540f5830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f8548009fb0 tx=0x7f8548009b30).stop 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 -- 172.21.15.175:0/645624622 shutdown_connections 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 --2- 172.21.15.175:0/645624622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540f5830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 -- 172.21.15.175:0/645624622 >> 172.21.15.175:0/645624622 conn(0x7f85540f0b70 msgr2=0x7f85540f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 -- 172.21.15.175:0/645624622 shutdown_connections 2023-12-06T03:24:55.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 -- 172.21.15.175:0/645624622 wait complete. 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 Processor -- start 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.503+0000 7f85597b9700 1 -- start start 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f85597b9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f85597b9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85540fb820 con 0x7f85540f5410 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49868/0 (socket says 172.21.15.175:49868) 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f8552ffd700 1 -- 172.21.15.175:0/1648733183 learned_addr learned my addr 172.21.15.175:0/1648733183 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f8552ffd700 1 -- 172.21.15.175:0/1648733183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548005040 con 0x7f85540f5410 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f8552ffd700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540fd3a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8548013510 tx=0x7f854800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:55.506 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8548009320 con 0x7f85540f5410 2023-12-06T03:24:55.507 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85540fb9c0 con 0x7f85540f5410 2023-12-06T03:24:55.507 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8548013f00 con 0x7f85540f5410 2023-12-06T03:24:55.507 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8548014d90 con 0x7f85540f5410 2023-12-06T03:24:55.507 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.504+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85540fbe50 con 0x7f85540f5410 2023-12-06T03:24:55.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.505+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f85480095f0 con 0x7f85540f5410 2023-12-06T03:24:55.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.505+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8554059100 con 0x7f85540f5410 2023-12-06T03:24:55.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.505+0000 7f853b7fe700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f853c02ff60 0x7f853c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:55.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.505+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8548041000 con 0x7f85540f5410 2023-12-06T03:24:55.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.506+0000 7f85527fc700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f853c02ff60 0x7f853c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:55.508 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.506+0000 7f85527fc700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f853c02ff60 0x7f853c032420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f85440096a0 tx=0x7f8544006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:55.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.510+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8548014420 con 0x7f85540f5410 2023-12-06T03:24:55.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.716+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f85540fae60 con 0x7f85540f5410 2023-12-06T03:24:55.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.716+0000 7f853b7fe700 1 -- 172.21.15.175:0/1648733183 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8548010230 con 0x7f85540f5410 2023-12-06T03:24:55.718 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:55.718 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.717+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f853c02ff60 msgr2=0x7f853c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.717+0000 7f85597b9700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f853c02ff60 0x7f853c032420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f85440096a0 tx=0x7f8544006b40).stop 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 msgr2=0x7f85540fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540fd3a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f8548013510 tx=0x7f854800b040).stop 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 shutdown_connections 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f853c02ff60 0x7f853c032420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 --2- 172.21.15.175:0/1648733183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85540f5410 0x7f85540fd3a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 >> 172.21.15.175:0/1648733183 conn(0x7f85540f0b70 msgr2=0x7f855405fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 shutdown_connections 2023-12-06T03:24:55.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:55.718+0000 7f85597b9700 1 -- 172.21.15.175:0/1648733183 wait complete. 2023-12-06T03:24:55.720 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:55.717271+0000 mon.smithi055 (mon.0) 189 : audit [DBG] from='client.? 172.21.15.175:0/1648733183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:57.149 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:57.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:24:57.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:56.436887+0000 mgr.smithi055.qjkkyb (mgr.14164) 108 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 -- 172.21.15.175:0/2776960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 msgr2=0x7fcbf40f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 --2- 172.21.15.175:0/2776960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf40f6930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fcbe8009fb0 tx=0x7fcbe8009b30).stop 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 -- 172.21.15.175:0/2776960128 shutdown_connections 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 --2- 172.21.15.175:0/2776960128 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf40f6930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 -- 172.21.15.175:0/2776960128 >> 172.21.15.175:0/2776960128 conn(0x7fcbf40f1c70 msgr2=0x7fcbf40f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 -- 172.21.15.175:0/2776960128 shutdown_connections 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 -- 172.21.15.175:0/2776960128 wait complete. 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.356+0000 7fcbfacf8700 1 Processor -- start 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbfacf8700 1 -- start start 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbfacf8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:58.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbfacf8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4069e20 con 0x7fcbf40f6510 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbf8a94700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbf8a94700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49882/0 (socket says 172.21.15.175:49882) 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbf8a94700 1 -- 172.21.15.175:0/3568529852 learned_addr learned my addr 172.21.15.175:0/3568529852 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbf8a94700 1 -- 172.21.15.175:0/3568529852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8005040 con 0x7fcbf40f6510 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.357+0000 7fcbf8a94700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf406b9a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fcbe8013510 tx=0x7fcbe800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbe8009320 con 0x7fcbf40f6510 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf4069fc0 con 0x7fcbf40f6510 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcbe8013f00 con 0x7fcbf40f6510 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbe8014d90 con 0x7fcbf40f6510 2023-12-06T03:24:58.360 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf406a450 con 0x7fcbf40f6510 2023-12-06T03:24:58.361 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fcbe8014420 con 0x7fcbf40f6510 2023-12-06T03:24:58.361 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.358+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf4059100 con 0x7fcbf40f6510 2023-12-06T03:24:58.361 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.359+0000 7fcbf17fa700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcbe402ff60 0x7fcbe4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:24:58.361 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.359+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fcbe8041000 con 0x7fcbf40f6510 2023-12-06T03:24:58.361 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.359+0000 7fcbf3fff700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcbe402ff60 0x7fcbe4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:24:58.361 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.359+0000 7fcbf3fff700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcbe402ff60 0x7fcbe4032420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fcbe00096a0 tx=0x7fcbe0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:24:58.365 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.364+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcbe80095f0 con 0x7fcbf40f6510 2023-12-06T03:24:58.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.567+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcbf40fbf60 con 0x7fcbf40f6510 2023-12-06T03:24:58.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.567+0000 7fcbf17fa700 1 -- 172.21.15.175:0/3568529852 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcbe801db70 con 0x7fcbf40f6510 2023-12-06T03:24:58.569 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:24:58.569 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:24:58.570 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcbe402ff60 msgr2=0x7fcbe4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcbe402ff60 0x7fcbe4032420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fcbe00096a0 tx=0x7fcbe0006b40).stop 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 msgr2=0x7fcbf406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf406b9a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fcbe8013510 tx=0x7fcbe800b040).stop 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 shutdown_connections 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcbe402ff60 0x7fcbe4032420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 --2- 172.21.15.175:0/3568529852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbf40f6510 0x7fcbf406b9a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 >> 172.21.15.175:0/3568529852 conn(0x7fcbf40f1c70 msgr2=0x7fcbf40f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 shutdown_connections 2023-12-06T03:24:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:24:58.569+0000 7fcbfacf8700 1 -- 172.21.15.175:0/3568529852 wait complete. 2023-12-06T03:24:58.572 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:24:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:24:58.437268+0000 mgr.smithi055.qjkkyb (mgr.14164) 109 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:24:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:24:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:24:58.568479+0000 mon.smithi055 (mon.0) 190 : audit [DBG] from='client.? 172.21.15.175:0/3568529852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:24:59.983 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:24:59.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 -- 172.21.15.175:0/1371886598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 msgr2=0x7f05200f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 --2- 172.21.15.175:0/1371886598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f05200f5830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f0510009fb0 tx=0x7f0510009b30).stop 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 -- 172.21.15.175:0/1371886598 shutdown_connections 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 --2- 172.21.15.175:0/1371886598 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f05200f5830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 -- 172.21.15.175:0/1371886598 >> 172.21.15.175:0/1371886598 conn(0x7f05200f0b70 msgr2=0x7f05200f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 -- 172.21.15.175:0/1371886598 shutdown_connections 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.113+0000 7f0524b19700 1 -- 172.21.15.175:0/1371886598 wait complete. 2023-12-06T03:25:01.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f0524b19700 1 Processor -- start 2023-12-06T03:25:01.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f0524b19700 1 -- start start 2023-12-06T03:25:01.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f0524b19700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f0520109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:01.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f0524b19700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520060240 con 0x7f05200f5410 2023-12-06T03:25:01.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f051e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f0520109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:01.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f051e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f0520109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49892/0 (socket says 172.21.15.175:49892) 2023-12-06T03:25:01.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.114+0000 7f051e59c700 1 -- 172.21.15.175:0/2879313452 learned_addr learned my addr 172.21.15.175:0/2879313452 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:01.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f051e59c700 1 -- 172.21.15.175:0/2879313452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0510005040 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f051e59c700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f0520109430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f0510013510 tx=0x7f051000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0510009320 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0520060440 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0510013f00 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0510014d90 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.115+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0520109ae0 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.116+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520045ad0 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.116+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f05100095f0 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.116+0000 7f050effd700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f050802ff60 0x7f0508032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.116+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f05100410a0 con 0x7f05200f5410 2023-12-06T03:25:01.118 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.116+0000 7f051dd9b700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f050802ff60 0x7f0508032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:01.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.117+0000 7f051dd9b700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f050802ff60 0x7f0508032420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f05140096a0 tx=0x7f0514006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:01.122 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.121+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f051001d7f0 con 0x7f05200f5410 2023-12-06T03:25:01.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.330+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f05200fae60 con 0x7f05200f5410 2023-12-06T03:25:01.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.330+0000 7f050effd700 1 -- 172.21.15.175:0/2879313452 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0510010230 con 0x7f05200f5410 2023-12-06T03:25:01.332 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:01.332 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f050802ff60 msgr2=0x7f0508032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f050802ff60 0x7f0508032420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f05140096a0 tx=0x7f0514006b40).stop 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 msgr2=0x7f0520109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f0520109430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f0510013510 tx=0x7f051000b040).stop 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 shutdown_connections 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f050802ff60 0x7f0508032420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 --2- 172.21.15.175:0/2879313452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f05200f5410 0x7f0520109430 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0x7f0510013510 tx=0x7f051000b040).stop 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 >> 172.21.15.175:0/2879313452 conn(0x7f05200f0b70 msgr2=0x7f05200f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:01.333 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 shutdown_connections 2023-12-06T03:25:01.334 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:01.332+0000 7f0524b19700 1 -- 172.21.15.175:0/2879313452 wait complete. 2023-12-06T03:25:01.334 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:00.437516+0000 mgr.smithi055.qjkkyb (mgr.14164) 110 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:01.331531+0000 mon.smithi055 (mon.0) 191 : audit [DBG] from='client.? 172.21.15.175:0/2879313452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:02.679 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:02.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:03.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 -- 172.21.15.175:0/2239548521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 msgr2=0x7f6a940f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 --2- 172.21.15.175:0/2239548521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a940f5830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f6a84009fb0 tx=0x7f6a84009b30).stop 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 -- 172.21.15.175:0/2239548521 shutdown_connections 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 --2- 172.21.15.175:0/2239548521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a940f5830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 -- 172.21.15.175:0/2239548521 >> 172.21.15.175:0/2239548521 conn(0x7f6a940f0b70 msgr2=0x7f6a940f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 -- 172.21.15.175:0/2239548521 shutdown_connections 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 -- 172.21.15.175:0/2239548521 wait complete. 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 Processor -- start 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.892+0000 7f6a98c50700 1 -- start start 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a98c50700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a94109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:03.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a98c50700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94060240 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a9259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a94109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a9259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a94109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49920/0 (socket says 172.21.15.175:49920) 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a9259c700 1 -- 172.21.15.175:0/484390540 learned_addr learned my addr 172.21.15.175:0/484390540 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a9259c700 1 -- 172.21.15.175:0/484390540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a84005040 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a9259c700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a94109430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f6a84013510 tx=0x7f6a8400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a84009320 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a94060440 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a84013f00 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.893+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a84014d90 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.894+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94109ae0 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.894+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a94045ad0 con 0x7f6a940f5410 2023-12-06T03:25:03.896 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.894+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6a840095f0 con 0x7f6a940f5410 2023-12-06T03:25:03.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.894+0000 7f6a8affd700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6a8002ff60 0x7f6a80032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:03.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.895+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6a84041000 con 0x7f6a940f5410 2023-12-06T03:25:03.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.895+0000 7f6a91d9b700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6a8002ff60 0x7f6a80032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:03.897 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.895+0000 7f6a91d9b700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6a8002ff60 0x7f6a80032420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6a7c0096a0 tx=0x7f6a7c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:03.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:03.899+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6a84014420 con 0x7f6a940f5410 2023-12-06T03:25:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:02.437780+0000 mgr.smithi055.qjkkyb (mgr.14164) 111 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:04.104 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.103+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6a940fae60 con 0x7f6a940f5410 2023-12-06T03:25:04.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.104+0000 7f6a8affd700 1 -- 172.21.15.175:0/484390540 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6a84010230 con 0x7f6a940f5410 2023-12-06T03:25:04.105 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:04.105 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:04.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6a8002ff60 msgr2=0x7f6a80032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:04.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6a8002ff60 0x7f6a80032420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f6a7c0096a0 tx=0x7f6a7c006b40).stop 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 msgr2=0x7f6a94109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a94109430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f6a84013510 tx=0x7f6a8400b040).stop 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 shutdown_connections 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6a8002ff60 0x7f6a80032420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 --2- 172.21.15.175:0/484390540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a940f5410 0x7f6a94109430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 >> 172.21.15.175:0/484390540 conn(0x7f6a940f0b70 msgr2=0x7f6a940f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 shutdown_connections 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:04.105+0000 7f6a98c50700 1 -- 172.21.15.175:0/484390540 wait complete. 2023-12-06T03:25:04.107 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:04.104813+0000 mon.smithi055 (mon.0) 192 : audit [DBG] from='client.? 172.21.15.175:0/484390540' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:05.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:04.438113+0000 mgr.smithi055.qjkkyb (mgr.14164) 112 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:05.497 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:05.497 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:06.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 -- 172.21.15.175:0/1793682325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 msgr2=0x7f12fc0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:06.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 --2- 172.21.15.175:0/1793682325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc0f5830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f12ec009fb0 tx=0x7f12ec009b30).stop 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 -- 172.21.15.175:0/1793682325 shutdown_connections 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 --2- 172.21.15.175:0/1793682325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc0f5830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 -- 172.21.15.175:0/1793682325 >> 172.21.15.175:0/1793682325 conn(0x7f12fc0f0b70 msgr2=0x7f12fc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 -- 172.21.15.175:0/1793682325 shutdown_connections 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.712+0000 7f1301c24700 1 -- 172.21.15.175:0/1793682325 wait complete. 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f1301c24700 1 Processor -- start 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f1301c24700 1 -- start start 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f1301c24700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f1301c24700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc060240 con 0x7f12fc0f5410 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f12fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f12fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40006/0 (socket says 172.21.15.175:40006) 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.713+0000 7f12fb7fe700 1 -- 172.21.15.175:0/3334010103 learned_addr learned my addr 172.21.15.175:0/3334010103 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f12fb7fe700 1 -- 172.21.15.175:0/3334010103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec005040 con 0x7f12fc0f5410 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f12fb7fe700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc109430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f12ec013510 tx=0x7f12ec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:06.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12ec009320 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12ec013f00 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc060440 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12ec014d90 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.714+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc109ae0 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.715+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f12ec0095f0 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.715+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc045ad0 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.715+0000 7f12f3fff700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12e802ffb0 0x7f12e8032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.715+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f12ec040fe0 con 0x7f12fc0f5410 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.715+0000 7f12faffd700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12e802ffb0 0x7f12e8032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:06.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.716+0000 7f12faffd700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12e802ffb0 0x7f12e8032470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f12e40096a0 tx=0x7f12e4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:06.721 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.720+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f12ec014420 con 0x7f12fc0f5410 2023-12-06T03:25:06.928 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.926+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f12fc0fae60 con 0x7f12fc0f5410 2023-12-06T03:25:06.928 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.927+0000 7f12f3fff700 1 -- 172.21.15.175:0/3334010103 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f12ec01d7f0 con 0x7f12fc0f5410 2023-12-06T03:25:06.929 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:06.929 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12e802ffb0 msgr2=0x7f12e8032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12e802ffb0 0x7f12e8032470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f12e40096a0 tx=0x7f12e4006b40).stop 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 msgr2=0x7f12fc109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc109430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f12ec013510 tx=0x7f12ec00b040).stop 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 shutdown_connections 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12e802ffb0 0x7f12e8032470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:06.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 --2- 172.21.15.175:0/3334010103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12fc0f5410 0x7f12fc109430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:06.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 >> 172.21.15.175:0/3334010103 conn(0x7f12fc0f0b70 msgr2=0x7f12fc0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:06.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 shutdown_connections 2023-12-06T03:25:06.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:06.929+0000 7f1301c24700 1 -- 172.21.15.175:0/3334010103 wait complete. 2023-12-06T03:25:06.931 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:06.438444+0000 mgr.smithi055.qjkkyb (mgr.14164) 113 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:06.928262+0000 mon.smithi055 (mon.0) 193 : audit [DBG] from='client.? 172.21.15.175:0/3334010103' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:08.307 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:08.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.462+0000 7ffaec48d700 1 -- 172.21.15.175:0/2491239831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 msgr2=0x7ffae40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.462+0000 7ffaec48d700 1 --2- 172.21.15.175:0/2491239831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae40f5830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7ffad8009fb0 tx=0x7ffad8009b30).stop 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 -- 172.21.15.175:0/2491239831 shutdown_connections 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 --2- 172.21.15.175:0/2491239831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae40f5830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 -- 172.21.15.175:0/2491239831 >> 172.21.15.175:0/2491239831 conn(0x7ffae40f0b70 msgr2=0x7ffae40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 -- 172.21.15.175:0/2491239831 shutdown_connections 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 -- 172.21.15.175:0/2491239831 wait complete. 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 Processor -- start 2023-12-06T03:25:09.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 -- start start 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae41093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.463+0000 7ffaec48d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4060220 con 0x7ffae40f5410 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaea229700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae41093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaea229700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae41093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40012/0 (socket says 172.21.15.175:40012) 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaea229700 1 -- 172.21.15.175:0/1838668164 learned_addr learned my addr 172.21.15.175:0/1838668164 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaea229700 1 -- 172.21.15.175:0/1838668164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad8005040 con 0x7ffae40f5410 2023-12-06T03:25:09.466 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaea229700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae41093f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ffad8013510 tx=0x7ffad800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffad8009320 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae4060420 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffad8013f00 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffad8014d90 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.464+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae4109aa0 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.465+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ffad80095f0 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.465+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae4059100 con 0x7ffae40f5410 2023-12-06T03:25:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.465+0000 7ffad6ffd700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:09.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.465+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ffad8041090 con 0x7ffae40f5410 2023-12-06T03:25:09.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.465+0000 7ffae9a28700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:09.468 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.466+0000 7ffae9a28700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ffae00096a0 tx=0x7ffae0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:09.471 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.470+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ffad8014420 con 0x7ffae40f5410 2023-12-06T03:25:09.677 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.676+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffae4045ad0 con 0x7ffae40f5410 2023-12-06T03:25:09.678 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.677+0000 7ffad6ffd700 1 -- 172.21.15.175:0/1838668164 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ffad801d7f0 con 0x7ffae40f5410 2023-12-06T03:25:09.678 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:09.679 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 msgr2=0x7ffad0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7ffae00096a0 tx=0x7ffae0006b40).stop 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 msgr2=0x7ffae41093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae41093f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ffad8013510 tx=0x7ffad800b040).stop 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 shutdown_connections 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 --2- 172.21.15.175:0/1838668164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae40f5410 0x7ffae41093f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:09.680 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 >> 172.21.15.175:0/1838668164 conn(0x7ffae40f0b70 msgr2=0x7ffae40f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 shutdown_connections 2023-12-06T03:25:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:09.679+0000 7ffaec48d700 1 -- 172.21.15.175:0/1838668164 wait complete. 2023-12-06T03:25:09.681 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:08.438815+0000 mgr.smithi055.qjkkyb (mgr.14164) 114 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:10.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:09.678108+0000 mon.smithi055 (mon.0) 194 : audit [DBG] from='client.? 172.21.15.175:0/1838668164' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:11.066 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:11.066 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:10.439161+0000 mgr.smithi055.qjkkyb (mgr.14164) 115 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 -- 172.21.15.175:0/2398421890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c40f5410 msgr2=0x7f41c40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 --2- 172.21.15.175:0/2398421890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c40f5410 0x7f41c40f5830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f41b4009fb0 tx=0x7f41b4009b30).stop 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 -- 172.21.15.175:0/2398421890 shutdown_connections 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 --2- 172.21.15.175:0/2398421890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c40f5410 0x7f41c40f5830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 -- 172.21.15.175:0/2398421890 >> 172.21.15.175:0/2398421890 conn(0x7f41c40f0b70 msgr2=0x7f41c40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 -- 172.21.15.175:0/2398421890 shutdown_connections 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.221+0000 7f41cb799700 1 -- 172.21.15.175:0/2398421890 wait complete. 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41cb799700 1 Processor -- start 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41cb799700 1 -- start start 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41cb799700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 0x7f41c4109580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41cb799700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4109a70 con 0x7f41c4060200 2023-12-06T03:25:12.224 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41c9535700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 0x7f41c4109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41c9535700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 0x7f41c4109580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40036/0 (socket says 172.21.15.175:40036) 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41c9535700 1 -- 172.21.15.175:0/2951922511 learned_addr learned my addr 172.21.15.175:0/2951922511 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41c9535700 1 -- 172.21.15.175:0/2951922511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4005040 con 0x7f41c4060200 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.222+0000 7f41c9535700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 0x7f41c4109580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f41b4009f80 tx=0x7f41b4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.223+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b40136c0 con 0x7f41c4060200 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.223+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c4109bb0 con 0x7f41c4060200 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.223+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f41b4013e70 con 0x7f41c4060200 2023-12-06T03:25:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.223+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b4014420 con 0x7f41c4060200 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.223+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c4109fe0 con 0x7f41c4060200 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.224+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f41b4025060 con 0x7f41c4060200 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.224+0000 7f41b9ffb700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f41b002ff60 0x7f41b0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.224+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f41b4040f40 con 0x7f41c4060200 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.224+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c4059100 con 0x7f41c4060200 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.224+0000 7f41c8d34700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f41b002ff60 0x7f41b0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:12.226 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.224+0000 7f41c8d34700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f41b002ff60 0x7f41b0032420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f41c00096a0 tx=0x7f41c0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:12.230 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.229+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f41b4028440 con 0x7f41c4060200 2023-12-06T03:25:12.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.436+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f41c4045ad0 con 0x7f41c4060200 2023-12-06T03:25:12.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.437+0000 7f41b9ffb700 1 -- 172.21.15.175:0/2951922511 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f41b4004490 con 0x7f41c4060200 2023-12-06T03:25:12.438 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:12.438 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:12.439 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f41b002ff60 msgr2=0x7f41b0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f41b002ff60 0x7f41b0032420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f41c00096a0 tx=0x7f41c0006b40).stop 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 msgr2=0x7f41c4109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 0x7f41c4109580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f41b4009f80 tx=0x7f41b4009b30).stop 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 shutdown_connections 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f41b002ff60 0x7f41b0032420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 --2- 172.21.15.175:0/2951922511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41c4060200 0x7f41c4109580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 >> 172.21.15.175:0/2951922511 conn(0x7f41c40f0b70 msgr2=0x7f41c40f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 shutdown_connections 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:12.438+0000 7f41cb799700 1 -- 172.21.15.175:0/2951922511 wait complete. 2023-12-06T03:25:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:12.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:12.437792+0000 mon.smithi055 (mon.0) 195 : audit [DBG] from='client.? 172.21.15.175:0/2951922511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:13.870 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:13.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:12.439398+0000 mgr.smithi055.qjkkyb (mgr.14164) 116 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 -- 172.21.15.175:0/2816353490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 msgr2=0x7fadc80f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 --2- 172.21.15.175:0/2816353490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc80f6930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fadbc009fb0 tx=0x7fadbc009b30).stop 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 -- 172.21.15.175:0/2816353490 shutdown_connections 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 --2- 172.21.15.175:0/2816353490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc80f6930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 -- 172.21.15.175:0/2816353490 >> 172.21.15.175:0/2816353490 conn(0x7fadc80f1c70 msgr2=0x7fadc80f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 -- 172.21.15.175:0/2816353490 shutdown_connections 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.106+0000 7fadcf02c700 1 -- 172.21.15.175:0/2816353490 wait complete. 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadcf02c700 1 Processor -- start 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadcf02c700 1 -- start start 2023-12-06T03:25:15.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadcf02c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadcf02c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8069e20 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadccdc8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadccdc8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40054/0 (socket says 172.21.15.175:40054) 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.107+0000 7fadccdc8700 1 -- 172.21.15.175:0/1964727950 learned_addr learned my addr 172.21.15.175:0/1964727950 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadccdc8700 1 -- 172.21.15.175:0/1964727950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc005040 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadccdc8700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc806b9a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fadbc013510 tx=0x7fadbc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadbc009320 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc8069fc0 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fadbc013f00 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadbc014d90 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.108+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc806a450 con 0x7fadc80f6510 2023-12-06T03:25:15.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.109+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fadbc014420 con 0x7fadc80f6510 2023-12-06T03:25:15.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.109+0000 7fadc57fa700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fadb802ff60 0x7fadb8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:15.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.109+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc8059100 con 0x7fadc80f6510 2023-12-06T03:25:15.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.109+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fadbc041080 con 0x7fadc80f6510 2023-12-06T03:25:15.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.109+0000 7fadc7fff700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fadb802ff60 0x7fadb8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:15.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.110+0000 7fadc7fff700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fadb802ff60 0x7fadb8032420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fadb40096a0 tx=0x7fadb4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:15.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.114+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fadbc0095f0 con 0x7fadc80f6510 2023-12-06T03:25:15.324 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.322+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fadc80fbf60 con 0x7fadc80f6510 2023-12-06T03:25:15.324 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.323+0000 7fadc57fa700 1 -- 172.21.15.175:0/1964727950 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fadbc01db70 con 0x7fadc80f6510 2023-12-06T03:25:15.325 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:15.325 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.324+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fadb802ff60 msgr2=0x7fadb8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.324+0000 7fadcf02c700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fadb802ff60 0x7fadb8032420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fadb40096a0 tx=0x7fadb4006b40).stop 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.324+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 msgr2=0x7fadc806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.324+0000 7fadcf02c700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc806b9a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fadbc013510 tx=0x7fadbc00b040).stop 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.324+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 shutdown_connections 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.325+0000 7fadcf02c700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fadb802ff60 0x7fadb8032420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.325+0000 7fadcf02c700 1 --2- 172.21.15.175:0/1964727950 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadc80f6510 0x7fadc806b9a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.325+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 >> 172.21.15.175:0/1964727950 conn(0x7fadc80f1c70 msgr2=0x7fadc80f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.325+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 shutdown_connections 2023-12-06T03:25:15.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:15.325+0000 7fadcf02c700 1 -- 172.21.15.175:0/1964727950 wait complete. 2023-12-06T03:25:15.327 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:15.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:14.439653+0000 mgr.smithi055.qjkkyb (mgr.14164) 117 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:16.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:16 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:15.324139+0000 mon.smithi055 (mon.0) 196 : audit [DBG] from='client.? 172.21.15.175:0/1964727950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:16.708 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:16.708 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:17.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:16.439981+0000 mgr.smithi055.qjkkyb (mgr.14164) 118 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:17.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 -- 172.21.15.175:0/315540955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 msgr2=0x7f79f40f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 --2- 172.21.15.175:0/315540955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f40f6930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f79f0009fb0 tx=0x7f79f0009b30).stop 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 -- 172.21.15.175:0/315540955 shutdown_connections 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 --2- 172.21.15.175:0/315540955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f40f6930 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 -- 172.21.15.175:0/315540955 >> 172.21.15.175:0/315540955 conn(0x7f79f40f1c70 msgr2=0x7f79f40f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 -- 172.21.15.175:0/315540955 shutdown_connections 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 -- 172.21.15.175:0/315540955 wait complete. 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 Processor -- start 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.836+0000 7f79fc542700 1 -- start start 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fc542700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f4106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fc542700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4106c20 con 0x7f79f40f6510 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fa2de700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f4106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fa2de700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f4106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58632/0 (socket says 172.21.15.175:58632) 2023-12-06T03:25:17.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fa2de700 1 -- 172.21.15.175:0/1761394289 learned_addr learned my addr 172.21.15.175:0/1761394289 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fa2de700 1 -- 172.21.15.175:0/1761394289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0005040 con 0x7f79f40f6510 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fa2de700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f4106730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f79f0013510 tx=0x7f79f000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79f0009320 con 0x7f79f40f6510 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f4106e20 con 0x7f79f40f6510 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79f0013f00 con 0x7f79f40f6510 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79f0014d90 con 0x7f79f40f6510 2023-12-06T03:25:17.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.837+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f41072b0 con 0x7f79f40f6510 2023-12-06T03:25:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.838+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f79f00095f0 con 0x7f79f40f6510 2023-12-06T03:25:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.838+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f4045ad0 con 0x7f79f40f6510 2023-12-06T03:25:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.838+0000 7f79eaffd700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f79e002ff60 0x7f79e0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.838+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f79f00410c0 con 0x7f79f40f6510 2023-12-06T03:25:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.839+0000 7f79f9add700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f79e002ff60 0x7f79e0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.839+0000 7f79f9add700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f79e002ff60 0x7f79e0032420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f79e40096a0 tx=0x7f79e4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:17.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:17.843+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f79f0014420 con 0x7f79f40f6510 2023-12-06T03:25:18.051 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.050+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f79f40fbf60 con 0x7f79f40f6510 2023-12-06T03:25:18.052 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.050+0000 7f79eaffd700 1 -- 172.21.15.175:0/1761394289 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f79f0010230 con 0x7f79f40f6510 2023-12-06T03:25:18.052 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:18.052 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:18.053 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f79e002ff60 msgr2=0x7f79e0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:18.053 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f79e002ff60 0x7f79e0032420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f79e40096a0 tx=0x7f79e4006b40).stop 2023-12-06T03:25:18.053 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 msgr2=0x7f79f4106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:18.053 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f4106730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f79f0013510 tx=0x7f79f000b040).stop 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 shutdown_connections 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f79e002ff60 0x7f79e0032420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 --2- 172.21.15.175:0/1761394289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f79f40f6510 0x7f79f4106730 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 >> 172.21.15.175:0/1761394289 conn(0x7f79f40f1c70 msgr2=0x7f79f40f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 shutdown_connections 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:18.052+0000 7f79fc542700 1 -- 172.21.15.175:0/1761394289 wait complete. 2023-12-06T03:25:18.054 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:18.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:18 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:18.051673+0000 mon.smithi055 (mon.0) 197 : audit [DBG] from='client.? 172.21.15.175:0/1761394289' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:19.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:18.440317+0000 mgr.smithi055.qjkkyb (mgr.14164) 119 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:19.458 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:19.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:20.650 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 -- 172.21.15.175:0/2413002749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 msgr2=0x7fdd080f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 --2- 172.21.15.175:0/2413002749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd080f6930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fdcfc009fb0 tx=0x7fdcfc009b30).stop 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 -- 172.21.15.175:0/2413002749 shutdown_connections 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 --2- 172.21.15.175:0/2413002749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd080f6930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 -- 172.21.15.175:0/2413002749 >> 172.21.15.175:0/2413002749 conn(0x7fdd080f1c70 msgr2=0x7fdd080f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 -- 172.21.15.175:0/2413002749 shutdown_connections 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 -- 172.21.15.175:0/2413002749 wait complete. 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.647+0000 7fdd0ee59700 1 Processor -- start 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.648+0000 7fdd0ee59700 1 -- start start 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.648+0000 7fdd0ee59700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd08106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.648+0000 7fdd0ee59700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08106c20 con 0x7fdd080f6510 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.648+0000 7fdd0cbf5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd08106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.648+0000 7fdd0cbf5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd08106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58656/0 (socket says 172.21.15.175:58656) 2023-12-06T03:25:20.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.648+0000 7fdd0cbf5700 1 -- 172.21.15.175:0/224272999 learned_addr learned my addr 172.21.15.175:0/224272999 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd0cbf5700 1 -- 172.21.15.175:0/224272999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc005040 con 0x7fdd080f6510 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd0cbf5700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd08106730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdcfc013510 tx=0x7fdcfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcfc009320 con 0x7fdd080f6510 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd08106e20 con 0x7fdd080f6510 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcfc013f00 con 0x7fdd080f6510 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdcfc014d90 con 0x7fdd080f6510 2023-12-06T03:25:20.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.649+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd081072b0 con 0x7fdd080f6510 2023-12-06T03:25:20.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.650+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fdcfc0095f0 con 0x7fdd080f6510 2023-12-06T03:25:20.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.650+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd08045ad0 con 0x7fdd080f6510 2023-12-06T03:25:20.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.650+0000 7fdd057fa700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdcf802ff60 0x7fdcf8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:20.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.650+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fdcfc041000 con 0x7fdd080f6510 2023-12-06T03:25:20.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.650+0000 7fdd07fff700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdcf802ff60 0x7fdcf8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:20.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.651+0000 7fdd07fff700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdcf802ff60 0x7fdcf8032420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fdcf40096a0 tx=0x7fdcf4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:20.656 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.655+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdcfc01d7f0 con 0x7fdd080f6510 2023-12-06T03:25:20.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.859+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdd080fbf60 con 0x7fdd080f6510 2023-12-06T03:25:20.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.859+0000 7fdd057fa700 1 -- 172.21.15.175:0/224272999 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fdcfc010230 con 0x7fdd080f6510 2023-12-06T03:25:20.861 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:20.861 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdcf802ff60 msgr2=0x7fdcf8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdcf802ff60 0x7fdcf8032420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fdcf40096a0 tx=0x7fdcf4006b40).stop 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 msgr2=0x7fdd08106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd08106730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdcfc013510 tx=0x7fdcfc00b040).stop 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 shutdown_connections 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdcf802ff60 0x7fdcf8032420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 --2- 172.21.15.175:0/224272999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd080f6510 0x7fdd08106730 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 >> 172.21.15.175:0/224272999 conn(0x7fdd080f1c70 msgr2=0x7fdd080f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 shutdown_connections 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:20.861+0000 7fdd0ee59700 1 -- 172.21.15.175:0/224272999 wait complete. 2023-12-06T03:25:20.863 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:20.440687+0000 mgr.smithi055.qjkkyb (mgr.14164) 120 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:20.860462+0000 mon.smithi055 (mon.0) 198 : audit [DBG] from='client.? 172.21.15.175:0/224272999' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:22.252 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:22.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:23.456 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 -- 172.21.15.175:0/2175250559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 msgr2=0x7f28ec0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:23.456 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 --2- 172.21.15.175:0/2175250559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec0f55c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f28e0009fb0 tx=0x7f28e0009b30).stop 2023-12-06T03:25:23.456 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 -- 172.21.15.175:0/2175250559 shutdown_connections 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 --2- 172.21.15.175:0/2175250559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec0f55c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 -- 172.21.15.175:0/2175250559 >> 172.21.15.175:0/2175250559 conn(0x7f28ec0f09a0 msgr2=0x7f28ec0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 -- 172.21.15.175:0/2175250559 shutdown_connections 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.454+0000 7f28f1bae700 1 -- 172.21.15.175:0/2175250559 wait complete. 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.455+0000 7f28f1bae700 1 Processor -- start 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.455+0000 7f28f1bae700 1 -- start start 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.455+0000 7f28f1bae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec1064a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:23.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.455+0000 7f28f1bae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec106990 con 0x7f28ec0f51a0 2023-12-06T03:25:23.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f0bac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec1064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:23.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f0bac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec1064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58680/0 (socket says 172.21.15.175:58680) 2023-12-06T03:25:23.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f0bac700 1 -- 172.21.15.175:0/3586371645 learned_addr learned my addr 172.21.15.175:0/3586371645 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:23.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f0bac700 1 -- 172.21.15.175:0/3586371645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0005040 con 0x7f28ec0f51a0 2023-12-06T03:25:23.458 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f0bac700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec1064a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f28e0009830 tx=0x7f28e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28e0009320 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec106b90 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28e00136c0 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28e001ee90 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.456+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec107020 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.457+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f28e0014420 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.457+0000 7f28e97fa700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f28dc02ff10 0x7f28dc0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.457+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f28e00411d0 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.457+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec045ad0 con 0x7f28ec0f51a0 2023-12-06T03:25:23.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.457+0000 7f28ebfff700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f28dc02ff10 0x7f28dc0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:23.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.458+0000 7f28ebfff700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f28dc02ff10 0x7f28dc0323d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f28d80096a0 tx=0x7f28d8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:23.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.462+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f28e001c320 con 0x7f28ec0f51a0 2023-12-06T03:25:23.667 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.666+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f28ec107d60 con 0x7f28ec0f51a0 2023-12-06T03:25:23.667 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.666+0000 7f28e97fa700 1 -- 172.21.15.175:0/3586371645 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f28e001c320 con 0x7f28ec0f51a0 2023-12-06T03:25:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:23.668 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f28dc02ff10 msgr2=0x7f28dc0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f28dc02ff10 0x7f28dc0323d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f28d80096a0 tx=0x7f28d8006b40).stop 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 msgr2=0x7f28ec1064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec1064a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f28e0009830 tx=0x7f28e000b040).stop 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 shutdown_connections 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f28dc02ff10 0x7f28dc0323d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 --2- 172.21.15.175:0/3586371645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28ec0f51a0 0x7f28ec1064a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 >> 172.21.15.175:0/3586371645 conn(0x7f28ec0f09a0 msgr2=0x7f28ec05fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 shutdown_connections 2023-12-06T03:25:23.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:23.668+0000 7f28f1bae700 1 -- 172.21.15.175:0/3586371645 wait complete. 2023-12-06T03:25:23.670 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:22.440935+0000 mgr.smithi055.qjkkyb (mgr.14164) 121 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:22.552428+0000 mon.smithi055 (mon.0) 199 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:25:23.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:22.555701+0000 mon.smithi055 (mon.0) 200 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:25:24.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:23.667383+0000 mon.smithi055 (mon.0) 201 : audit [DBG] from='client.? 172.21.15.175:0/3586371645' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:25.071 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:25.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:25.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:24.441291+0000 mgr.smithi055.qjkkyb (mgr.14164) 122 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 -- 172.21.15.175:0/4239150952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 msgr2=0x7f9de80f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:26.209 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 --2- 172.21.15.175:0/4239150952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f90f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f9dd8009fb0 tx=0x7f9dd8009b30).stop 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 -- 172.21.15.175:0/4239150952 shutdown_connections 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 --2- 172.21.15.175:0/4239150952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f90f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 -- 172.21.15.175:0/4239150952 >> 172.21.15.175:0/4239150952 conn(0x7f9de80f0b70 msgr2=0x7f9de80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 -- 172.21.15.175:0/4239150952 shutdown_connections 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 -- 172.21.15.175:0/4239150952 wait complete. 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 Processor -- start 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.206+0000 7f9dec843700 1 -- start start 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dec843700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dec843700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80f4ff0 con 0x7f9de8060280 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9de659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9de659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56290/0 (socket says 172.21.15.175:56290) 2023-12-06T03:25:26.210 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9de659c700 1 -- 172.21.15.175:0/3977307161 learned_addr learned my addr 172.21.15.175:0/3977307161 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9de659c700 1 -- 172.21.15.175:0/3977307161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8005040 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9de659c700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f6b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f9dd8013510 tx=0x7f9dd800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dd8009320 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de80f5190 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9dd8013f00 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dd8014dc0 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.207+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de80f5620 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.208+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9dd8014420 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.208+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de8059100 con 0x7f9de8060280 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.208+0000 7f9dd6ffd700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9dd002ff60 0x7f9dd0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:26.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.208+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9dd8041050 con 0x7f9de8060280 2023-12-06T03:25:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.209+0000 7f9de5d9b700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9dd002ff60 0x7f9dd0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:26.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.209+0000 7f9de5d9b700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9dd002ff60 0x7f9dd0032420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f9ddc0096a0 tx=0x7f9ddc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:26.215 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.214+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9dd8026bf0 con 0x7f9de8060280 2023-12-06T03:25:26.420 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.419+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9de80fbf50 con 0x7f9de8060280 2023-12-06T03:25:26.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.420+0000 7f9dd6ffd700 1 -- 172.21.15.175:0/3977307161 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9dd801cb10 con 0x7f9de8060280 2023-12-06T03:25:26.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:26.422 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:26.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9dd002ff60 msgr2=0x7f9dd0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:26.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9dd002ff60 0x7f9dd0032420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f9ddc0096a0 tx=0x7f9ddc006b40).stop 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 msgr2=0x7f9de80f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f6b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f9dd8013510 tx=0x7f9dd800b040).stop 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 shutdown_connections 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9dd002ff60 0x7f9dd0032420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 --2- 172.21.15.175:0/3977307161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9de8060280 0x7f9de80f6b70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 >> 172.21.15.175:0/3977307161 conn(0x7f9de80f0b70 msgr2=0x7f9de80f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 shutdown_connections 2023-12-06T03:25:26.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:26.422+0000 7f9dec843700 1 -- 172.21.15.175:0/3977307161 wait complete. 2023-12-06T03:25:26.425 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:26.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:26.421229+0000 mon.smithi055 (mon.0) 202 : audit [DBG] from='client.? 172.21.15.175:0/3977307161' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:27.816 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:27.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:26.441501+0000 mgr.smithi055.qjkkyb (mgr.14164) 123 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 -- 172.21.15.175:0/1218308717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 msgr2=0x7fd8940f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 --2- 172.21.15.175:0/1218308717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd8940f5830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fd884009fb0 tx=0x7fd884009b30).stop 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 -- 172.21.15.175:0/1218308717 shutdown_connections 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 --2- 172.21.15.175:0/1218308717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd8940f5830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 -- 172.21.15.175:0/1218308717 >> 172.21.15.175:0/1218308717 conn(0x7fd8940f0b70 msgr2=0x7fd8940f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 -- 172.21.15.175:0/1218308717 shutdown_connections 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.960+0000 7fd89bd9c700 1 -- 172.21.15.175:0/1218308717 wait complete. 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd89bd9c700 1 Processor -- start 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd89bd9c700 1 -- start start 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd89bd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd89410b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd89bd9c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd894107240 con 0x7fd8940f5410 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd899b38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd89410b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd899b38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd89410b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56320/0 (socket says 172.21.15.175:56320) 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd899b38700 1 -- 172.21.15.175:0/280443997 learned_addr learned my addr 172.21.15.175:0/280443997 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd899b38700 1 -- 172.21.15.175:0/280443997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884005040 con 0x7fd8940f5410 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd899b38700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd89410b850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd884013510 tx=0x7fd88400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd884009320 con 0x7fd8940f5410 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.961+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd894060220 con 0x7fd8940f5410 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.962+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd884013f00 con 0x7fd8940f5410 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.962+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd884014d90 con 0x7fd8940f5410 2023-12-06T03:25:28.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.962+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89410bd40 con 0x7fd8940f5410 2023-12-06T03:25:28.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.962+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd884014420 con 0x7fd8940f5410 2023-12-06T03:25:28.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.962+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd894045ad0 con 0x7fd8940f5410 2023-12-06T03:25:28.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.963+0000 7fd88a7fc700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd88002ff60 0x7fd880032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:28.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.963+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd884041000 con 0x7fd8940f5410 2023-12-06T03:25:28.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.963+0000 7fd899337700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd88002ff60 0x7fd880032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:28.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.963+0000 7fd899337700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd88002ff60 0x7fd880032420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd8900096a0 tx=0x7fd890006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:28.970 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:28.968+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd8840095f0 con 0x7fd8940f5410 2023-12-06T03:25:29.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.177+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd8940fae60 con 0x7fd8940f5410 2023-12-06T03:25:29.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.178+0000 7fd88a7fc700 1 -- 172.21.15.175:0/280443997 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd88401db70 con 0x7fd8940f5410 2023-12-06T03:25:29.179 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:29.179 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:29.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd88002ff60 msgr2=0x7fd880032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd88002ff60 0x7fd880032420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd8900096a0 tx=0x7fd890006b40).stop 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 msgr2=0x7fd89410b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd89410b850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd884013510 tx=0x7fd88400b040).stop 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 shutdown_connections 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd88002ff60 0x7fd880032420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 --2- 172.21.15.175:0/280443997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8940f5410 0x7fd89410b850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 >> 172.21.15.175:0/280443997 conn(0x7fd8940f0b70 msgr2=0x7fd8940f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 shutdown_connections 2023-12-06T03:25:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:29.179+0000 7fd89bd9c700 1 -- 172.21.15.175:0/280443997 wait complete. 2023-12-06T03:25:29.182 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:28.441846+0000 mgr.smithi055.qjkkyb (mgr.14164) 124 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:29.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:29.178704+0000 mon.smithi055 (mon.0) 203 : audit [DBG] from='client.? 172.21.15.175:0/280443997' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:30.570 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:30.570 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:31.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 -- 172.21.15.175:0/3571223411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 msgr2=0x7fb3a80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:31.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 --2- 172.21.15.175:0/3571223411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a80f5830 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb39c009fb0 tx=0x7fb39c009b30).stop 2023-12-06T03:25:31.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 -- 172.21.15.175:0/3571223411 shutdown_connections 2023-12-06T03:25:31.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 --2- 172.21.15.175:0/3571223411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a80f5830 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:31.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 -- 172.21.15.175:0/3571223411 >> 172.21.15.175:0/3571223411 conn(0x7fb3a80f0b70 msgr2=0x7fb3a80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:31.833 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 -- 172.21.15.175:0/3571223411 shutdown_connections 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.830+0000 7fb3afab8700 1 -- 172.21.15.175:0/3571223411 wait complete. 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3afab8700 1 Processor -- start 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3afab8700 1 -- start start 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3afab8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a81066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3afab8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a8106bd0 con 0x7fb3a80f5410 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3ad854700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a81066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3ad854700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a81066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56342/0 (socket says 172.21.15.175:56342) 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3ad854700 1 -- 172.21.15.175:0/116518155 learned_addr learned my addr 172.21.15.175:0/116518155 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3ad854700 1 -- 172.21.15.175:0/116518155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c005040 con 0x7fb3a80f5410 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.831+0000 7fb3ad854700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a81066e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb39c014000 tx=0x7fb39c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb39c0095f0 con 0x7fb3a80f5410 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb39c007e70 con 0x7fb3a80f5410 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a8106dd0 con 0x7fb3a80f5410 2023-12-06T03:25:31.834 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb39c014ea0 con 0x7fb3a80f5410 2023-12-06T03:25:31.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a8107260 con 0x7fb3a80f5410 2023-12-06T03:25:31.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb39c025070 con 0x7fb3a80f5410 2023-12-06T03:25:31.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.832+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a8045ad0 con 0x7fb3a80f5410 2023-12-06T03:25:31.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.833+0000 7fb39a7fc700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb39402ff60 0x7fb394032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:31.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.833+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb39c040970 con 0x7fb3a80f5410 2023-12-06T03:25:31.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.833+0000 7fb3ad053700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb39402ff60 0x7fb394032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:31.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.833+0000 7fb3ad053700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb39402ff60 0x7fb394032420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb3a40096a0 tx=0x7fb3a4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:31.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:31.839+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb39c0079c0 con 0x7fb3a80f5410 2023-12-06T03:25:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:30.442286+0000 mgr.smithi055.qjkkyb (mgr.14164) 125 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:32.045 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.045+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb3a80fae60 con 0x7fb3a80f5410 2023-12-06T03:25:32.046 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.046+0000 7fb39a7fc700 1 -- 172.21.15.175:0/116518155 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb39c00f300 con 0x7fb3a80f5410 2023-12-06T03:25:32.046 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:32.046 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb39402ff60 msgr2=0x7fb394032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb39402ff60 0x7fb394032420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fb3a40096a0 tx=0x7fb3a4006b40).stop 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 msgr2=0x7fb3a81066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a81066e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb39c014000 tx=0x7fb39c00b040).stop 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 shutdown_connections 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb39402ff60 0x7fb394032420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 --2- 172.21.15.175:0/116518155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3a80f5410 0x7fb3a81066e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.047+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 >> 172.21.15.175:0/116518155 conn(0x7fb3a80f0b70 msgr2=0x7fb3a805fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.048+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 shutdown_connections 2023-12-06T03:25:32.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:32.048+0000 7fb3afab8700 1 -- 172.21.15.175:0/116518155 wait complete. 2023-12-06T03:25:32.049 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:32.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:32 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:32.046021+0000 mon.smithi055 (mon.0) 204 : audit [DBG] from='client.? 172.21.15.175:0/116518155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:33.467 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:33.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:32.442670+0000 mgr.smithi055.qjkkyb (mgr.14164) 126 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:34.601 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 -- 172.21.15.175:0/682032585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 msgr2=0x7f80b40f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:34.601 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 --2- 172.21.15.175:0/682032585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b40f6930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f80a4009fb0 tx=0x7f80a4009b30).stop 2023-12-06T03:25:34.601 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 -- 172.21.15.175:0/682032585 shutdown_connections 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 --2- 172.21.15.175:0/682032585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b40f6930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 -- 172.21.15.175:0/682032585 >> 172.21.15.175:0/682032585 conn(0x7f80b40f1c70 msgr2=0x7f80b40f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 -- 172.21.15.175:0/682032585 shutdown_connections 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.599+0000 7f80bc761700 1 -- 172.21.15.175:0/682032585 wait complete. 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80bc761700 1 Processor -- start 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80bc761700 1 -- start start 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80bc761700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b4108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80bc761700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4108e00 con 0x7f80b40f6510 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80ba4fd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b4108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80ba4fd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b4108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:56368/0 (socket says 172.21.15.175:56368) 2023-12-06T03:25:34.602 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.600+0000 7f80ba4fd700 1 -- 172.21.15.175:0/1453393669 learned_addr learned my addr 172.21.15.175:0/1453393669 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80ba4fd700 1 -- 172.21.15.175:0/1453393669 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4005040 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80ba4fd700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b4108910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f80a4013510 tx=0x7f80a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80a4009320 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b4109000 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80a4013f00 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80a4014d90 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.601+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b4109490 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.602+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f80a4014420 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.602+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b4045ad0 con 0x7f80b40f6510 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.602+0000 7f80aaffd700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f80a002ff60 0x7f80a0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:34.604 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.602+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f80a4041000 con 0x7f80b40f6510 2023-12-06T03:25:34.605 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.602+0000 7f80b9cfc700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f80a002ff60 0x7f80a0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:34.605 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.603+0000 7f80b9cfc700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f80a002ff60 0x7f80a0032420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f80b00096a0 tx=0x7f80b0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:34.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.607+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f80a40095f0 con 0x7f80b40f6510 2023-12-06T03:25:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.817+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f80b410a340 con 0x7f80b40f6510 2023-12-06T03:25:34.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.817+0000 7f80aaffd700 1 -- 172.21.15.175:0/1453393669 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f80a401db70 con 0x7f80b40f6510 2023-12-06T03:25:34.820 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:34.820 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f80a002ff60 msgr2=0x7f80a0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f80a002ff60 0x7f80a0032420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f80b00096a0 tx=0x7f80b0006b40).stop 2023-12-06T03:25:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 msgr2=0x7f80b4108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:34.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b4108910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f80a4013510 tx=0x7f80a400b040).stop 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 shutdown_connections 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f80a002ff60 0x7f80a0032420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 --2- 172.21.15.175:0/1453393669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b40f6510 0x7f80b4108910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 >> 172.21.15.175:0/1453393669 conn(0x7f80b40f1c70 msgr2=0x7f80b40f26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 shutdown_connections 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:34.819+0000 7f80bc761700 1 -- 172.21.15.175:0/1453393669 wait complete. 2023-12-06T03:25:34.822 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:34.443058+0000 mgr.smithi055.qjkkyb (mgr.14164) 127 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:34.818530+0000 mon.smithi055 (mon.0) 205 : audit [DBG] from='client.? 172.21.15.175:0/1453393669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:36.218 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:36.218 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:37.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 -- 172.21.15.175:0/1361081548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 msgr2=0x7f57740f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 --2- 172.21.15.175:0/1361081548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f57740f6930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f5768009fb0 tx=0x7f5768009b30).stop 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 -- 172.21.15.175:0/1361081548 shutdown_connections 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 --2- 172.21.15.175:0/1361081548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f57740f6930 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 -- 172.21.15.175:0/1361081548 >> 172.21.15.175:0/1361081548 conn(0x7f57740f1c70 msgr2=0x7f57740f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 -- 172.21.15.175:0/1361081548 shutdown_connections 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 -- 172.21.15.175:0/1361081548 wait complete. 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 Processor -- start 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.316+0000 7f577921a700 1 -- start start 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f577921a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f5774106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f577921a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774106c20 con 0x7f57740f6510 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f5772d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f5774106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:37.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f5772d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f5774106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:39986/0 (socket says 172.21.15.175:39986) 2023-12-06T03:25:37.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f5772d9d700 1 -- 172.21.15.175:0/641230120 learned_addr learned my addr 172.21.15.175:0/641230120 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:37.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f5772d9d700 1 -- 172.21.15.175:0/641230120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768005040 con 0x7f57740f6510 2023-12-06T03:25:37.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f5772d9d700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f5774106730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f5768013510 tx=0x7f576800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:37.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5768009320 con 0x7f57740f6510 2023-12-06T03:25:37.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5774106e20 con 0x7f57740f6510 2023-12-06T03:25:37.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5768013f00 con 0x7f57740f6510 2023-12-06T03:25:37.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5768014d90 con 0x7f57740f6510 2023-12-06T03:25:37.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.317+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57741072b0 con 0x7f57740f6510 2023-12-06T03:25:37.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.318+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f57680263a0 con 0x7f57740f6510 2023-12-06T03:25:37.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.318+0000 7f575b7fe700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f575c02ff60 0x7f575c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:37.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.318+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f57680410c0 con 0x7f57740f6510 2023-12-06T03:25:37.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.318+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5774045ad0 con 0x7f57740f6510 2023-12-06T03:25:37.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.319+0000 7f577259c700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f575c02ff60 0x7f575c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:37.322 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.319+0000 7f577259c700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f575c02ff60 0x7f575c032420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f57640096a0 tx=0x7f5764006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:37.324 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.324+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5768012920 con 0x7f57740f6510 2023-12-06T03:25:37.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.529+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f57740fc050 con 0x7f57740f6510 2023-12-06T03:25:37.531 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.530+0000 7f575b7fe700 1 -- 172.21.15.175:0/641230120 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f576802b020 con 0x7f57740f6510 2023-12-06T03:25:37.531 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:37.531 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:37.532 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f575c02ff60 msgr2=0x7f575c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f575c02ff60 0x7f575c032420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f57640096a0 tx=0x7f5764006b40).stop 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 msgr2=0x7f5774106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f5774106730 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f5768013510 tx=0x7f576800b040).stop 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 shutdown_connections 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f575c02ff60 0x7f575c032420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 --2- 172.21.15.175:0/641230120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57740f6510 0x7f5774106730 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 >> 172.21.15.175:0/641230120 conn(0x7f57740f1c70 msgr2=0x7f57740f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 shutdown_connections 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:37.531+0000 7f577921a700 1 -- 172.21.15.175:0/641230120 wait complete. 2023-12-06T03:25:37.533 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:37.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:36.443337+0000 mgr.smithi055.qjkkyb (mgr.14164) 128 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:38.942 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:38.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:38.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:38 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:37.530848+0000 mon.smithi055 (mon.0) 206 : audit [DBG] from='client.? 172.21.15.175:0/641230120' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:38.443622+0000 mgr.smithi055.qjkkyb (mgr.14164) 129 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:40.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 -- 172.21.15.175:0/1956651150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 msgr2=0x7f567c069e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:40.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 --2- 172.21.15.175:0/1956651150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c069e60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f5670009fb0 tx=0x7f5670009b30).stop 2023-12-06T03:25:40.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 -- 172.21.15.175:0/1956651150 shutdown_connections 2023-12-06T03:25:40.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 --2- 172.21.15.175:0/1956651150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c069e60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:40.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 -- 172.21.15.175:0/1956651150 >> 172.21.15.175:0/1956651150 conn(0x7f567c0f1b90 msgr2=0x7f567c0f3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 -- 172.21.15.175:0/1956651150 shutdown_connections 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.139+0000 7f56834ec700 1 -- 172.21.15.175:0/1956651150 wait complete. 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f56834ec700 1 Processor -- start 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f56834ec700 1 -- start start 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f56834ec700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c108b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f56834ec700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c109050 con 0x7f567c06b960 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f5681288700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c108b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f5681288700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c108b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40008/0 (socket says 172.21.15.175:40008) 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f5681288700 1 -- 172.21.15.175:0/756378792 learned_addr learned my addr 172.21.15.175:0/756378792 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.140+0000 7f5681288700 1 -- 172.21.15.175:0/756378792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670005040 con 0x7f567c06b960 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.141+0000 7f5681288700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c108b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f5670013510 tx=0x7f567000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.141+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5670009320 con 0x7f567c06b960 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.141+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c109250 con 0x7f567c06b960 2023-12-06T03:25:40.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.141+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5670013f00 con 0x7f567c06b960 2023-12-06T03:25:40.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.141+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5670014dc0 con 0x7f567c06b960 2023-12-06T03:25:40.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.141+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c1096e0 con 0x7f567c06b960 2023-12-06T03:25:40.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.142+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f56700095f0 con 0x7f567c06b960 2023-12-06T03:25:40.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.142+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c045ad0 con 0x7f567c06b960 2023-12-06T03:25:40.144 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.142+0000 7f566dffb700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:40.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.142+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5670041020 con 0x7f567c06b960 2023-12-06T03:25:40.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.142+0000 7f5680a87700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:40.145 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.143+0000 7f5680a87700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f56780096a0 tx=0x7f5678006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:40.148 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.147+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5670014420 con 0x7f567c06b960 2023-12-06T03:25:40.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.351+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f567c05ea50 con 0x7f567c06b960 2023-12-06T03:25:40.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.351+0000 7f566dffb700 1 -- 172.21.15.175:0/756378792 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f567001c920 con 0x7f567c06b960 2023-12-06T03:25:40.353 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:40.353 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 msgr2=0x7f5668032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f56780096a0 tx=0x7f5678006b40).stop 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 msgr2=0x7f567c108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c108b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f5670013510 tx=0x7f567000b040).stop 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 shutdown_connections 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 --2- 172.21.15.175:0/756378792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f567c06b960 0x7f567c108b60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 >> 172.21.15.175:0/756378792 conn(0x7f567c0f1b90 msgr2=0x7f567c0f2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 shutdown_connections 2023-12-06T03:25:40.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:40.353+0000 7f56834ec700 1 -- 172.21.15.175:0/756378792 wait complete. 2023-12-06T03:25:40.356 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:40.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:40 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:40.352642+0000 mon.smithi055 (mon.0) 207 : audit [DBG] from='client.? 172.21.15.175:0/756378792' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:41.774 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:41.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:40.443910+0000 mgr.smithi055.qjkkyb (mgr.14164) 130 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:42.949 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 -- 172.21.15.175:0/2381102774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 msgr2=0x7efd0c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:42.949 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 --2- 172.21.15.175:0/2381102774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c0f5830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7efcfc009fb0 tx=0x7efcfc009b30).stop 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 -- 172.21.15.175:0/2381102774 shutdown_connections 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 --2- 172.21.15.175:0/2381102774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c0f5830 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 -- 172.21.15.175:0/2381102774 >> 172.21.15.175:0/2381102774 conn(0x7efd0c0f0b70 msgr2=0x7efd0c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 -- 172.21.15.175:0/2381102774 shutdown_connections 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 -- 172.21.15.175:0/2381102774 wait complete. 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.947+0000 7efd129d3700 1 Processor -- start 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd129d3700 1 -- start start 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd129d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c1093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd129d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c060220 con 0x7efd0c0f5410 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd0bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c1093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd0bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c1093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40030/0 (socket says 172.21.15.175:40030) 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd0bfff700 1 -- 172.21.15.175:0/3089198472 learned_addr learned my addr 172.21.15.175:0/3089198472 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:42.950 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd0bfff700 1 -- 172.21.15.175:0/3089198472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc005040 con 0x7efd0c0f5410 2023-12-06T03:25:42.951 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.948+0000 7efd0bfff700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c1093f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7efcfc013510 tx=0x7efcfc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.949+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcfc009320 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.949+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c060420 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.949+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c109aa0 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.949+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcfc013f00 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.949+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efcfc014d60 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.950+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7efcfc0263a0 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.950+0000 7efd08ff9700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efcf802ff60 0x7efcf8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.950+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7efcfc041fe0 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.950+0000 7efd0b7fe700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efcf802ff60 0x7efcf8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.950+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c059100 con 0x7efd0c0f5410 2023-12-06T03:25:42.952 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.950+0000 7efd0b7fe700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efcf802ff60 0x7efcf8032420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7efcf40096a0 tx=0x7efcf4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:42.956 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:42.955+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efcfc01d380 con 0x7efd0c0f5410 2023-12-06T03:25:43.160 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.159+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efd0c045ad0 con 0x7efd0c0f5410 2023-12-06T03:25:43.161 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.160+0000 7efd08ff9700 1 -- 172.21.15.175:0/3089198472 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efd0c045ad0 con 0x7efd0c0f5410 2023-12-06T03:25:43.161 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:43.162 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.161+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efcf802ff60 msgr2=0x7efcf8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.161+0000 7efd129d3700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efcf802ff60 0x7efcf8032420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7efcf40096a0 tx=0x7efcf4006b40).stop 2023-12-06T03:25:43.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 msgr2=0x7efd0c1093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c1093f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7efcfc013510 tx=0x7efcfc00b040).stop 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 shutdown_connections 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efcf802ff60 0x7efcf8032420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 --2- 172.21.15.175:0/3089198472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efd0c0f5410 0x7efd0c1093f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 >> 172.21.15.175:0/3089198472 conn(0x7efd0c0f0b70 msgr2=0x7efd0c0f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 shutdown_connections 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:43.162+0000 7efd129d3700 1 -- 172.21.15.175:0/3089198472 wait complete. 2023-12-06T03:25:43.167 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:42.444261+0000 mgr.smithi055.qjkkyb (mgr.14164) 131 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:43.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:43.160999+0000 mon.smithi055 (mon.0) 208 : audit [DBG] from='client.? 172.21.15.175:0/3089198472' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:44.506 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:44.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:44.444639+0000 mgr.smithi055.qjkkyb (mgr.14164) 132 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.663+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/3089524271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 msgr2=0x7fa8b40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 --2- 172.21.15.175:0/3089524271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b40f5830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fa8a4009fb0 tx=0x7fa8a4009b30).stop 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/3089524271 shutdown_connections 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 --2- 172.21.15.175:0/3089524271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b40f5830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/3089524271 >> 172.21.15.175:0/3089524271 conn(0x7fa8b40f0b70 msgr2=0x7fa8b40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/3089524271 shutdown_connections 2023-12-06T03:25:45.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/3089524271 wait complete. 2023-12-06T03:25:45.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 Processor -- start 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 -- start start 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b4106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.664+0000 7fa8ba0d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4106c20 con 0x7fa8b40f5410 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa8b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b4106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa8b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b4106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57502/0 (socket says 172.21.15.175:57502) 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa8b37fe700 1 -- 172.21.15.175:0/2520045388 learned_addr learned my addr 172.21.15.175:0/2520045388 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa8b37fe700 1 -- 172.21.15.175:0/2520045388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a4005040 con 0x7fa8b40f5410 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa8b37fe700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b4106730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fa8a4013510 tx=0x7fa8a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8a4009320 con 0x7fa8b40f5410 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b4106e20 con 0x7fa8b40f5410 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa8a4013f00 con 0x7fa8b40f5410 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.665+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8a4014d90 con 0x7fa8b40f5410 2023-12-06T03:25:45.668 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.666+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b41072b0 con 0x7fa8b40f5410 2023-12-06T03:25:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.666+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa8a40263a0 con 0x7fa8b40f5410 2023-12-06T03:25:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.666+0000 7fa89bfff700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa89c02ff60 0x7fa89c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.666+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa8a40410e0 con 0x7fa8b40f5410 2023-12-06T03:25:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.666+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b4045ad0 con 0x7fa8b40f5410 2023-12-06T03:25:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.667+0000 7fa8b2ffd700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa89c02ff60 0x7fa89c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.667+0000 7fa8b2ffd700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa89c02ff60 0x7fa89c032420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fa8a80096a0 tx=0x7fa8a8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:45.673 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.672+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa8a4012920 con 0x7fa8b40f5410 2023-12-06T03:25:45.881 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.879+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa8b40fae60 con 0x7fa8b40f5410 2023-12-06T03:25:45.881 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.879+0000 7fa89bfff700 1 -- 172.21.15.175:0/2520045388 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa8a402b020 con 0x7fa8b40f5410 2023-12-06T03:25:45.881 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:45.881 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa89c02ff60 msgr2=0x7fa89c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa89c02ff60 0x7fa89c032420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fa8a80096a0 tx=0x7fa8a8006b40).stop 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 msgr2=0x7fa8b4106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b4106730 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fa8a4013510 tx=0x7fa8a400b040).stop 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 shutdown_connections 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa89c02ff60 0x7fa89c032420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 --2- 172.21.15.175:0/2520045388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8b40f5410 0x7fa8b4106730 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:45.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 >> 172.21.15.175:0/2520045388 conn(0x7fa8b40f0b70 msgr2=0x7fa8b40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:45.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 shutdown_connections 2023-12-06T03:25:45.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:45.881+0000 7fa8ba0d0700 1 -- 172.21.15.175:0/2520045388 wait complete. 2023-12-06T03:25:45.884 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:46.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:45.880510+0000 mon.smithi055 (mon.0) 209 : audit [DBG] from='client.? 172.21.15.175:0/2520045388' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:47.263 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:47.264 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:47.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:46.444982+0000 mgr.smithi055.qjkkyb (mgr.14164) 133 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:48.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 -- 172.21.15.175:0/3330581780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 msgr2=0x7f03ac0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:48.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 --2- 172.21.15.175:0/3330581780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac0f55c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f03a0009fb0 tx=0x7f03a0009b30).stop 2023-12-06T03:25:48.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 -- 172.21.15.175:0/3330581780 shutdown_connections 2023-12-06T03:25:48.459 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 --2- 172.21.15.175:0/3330581780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac0f55c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:48.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 -- 172.21.15.175:0/3330581780 >> 172.21.15.175:0/3330581780 conn(0x7f03ac0f09a0 msgr2=0x7f03ac0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:48.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 -- 172.21.15.175:0/3330581780 shutdown_connections 2023-12-06T03:25:48.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.457+0000 7f03b1f1b700 1 -- 172.21.15.175:0/3330581780 wait complete. 2023-12-06T03:25:48.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.458+0000 7f03b1f1b700 1 Processor -- start 2023-12-06T03:25:48.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.458+0000 7f03b1f1b700 1 -- start start 2023-12-06T03:25:48.461 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b1f1b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac1053b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:48.461 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b1f1b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1058a0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.461 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b0f19700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac1053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b0f19700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac1053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57520/0 (socket says 172.21.15.175:57520) 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b0f19700 1 -- 172.21.15.175:0/4094270795 learned_addr learned my addr 172.21.15.175:0/4094270795 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b0f19700 1 -- 172.21.15.175:0/4094270795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0005040 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b0f19700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac1053b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f03a0013510 tx=0x7f03a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03a0009320 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03a0013f00 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac105aa0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03a0014dc0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.459+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac105f30 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.460+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f03a00095f0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.460+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac045ad0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.460+0000 7f03a97fa700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f039c02ffb0 0x7f039c032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:48.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.460+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f03a0041070 con 0x7f03ac0f51a0 2023-12-06T03:25:48.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.461+0000 7f03abfff700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f039c02ffb0 0x7f039c032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:48.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.461+0000 7f03abfff700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f039c02ffb0 0x7f039c032470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f03980096a0 tx=0x7f0398006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:48.467 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.465+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f03a006d2b0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.672 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.671+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f03ac106de0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.672 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.671+0000 7f03a97fa700 1 -- 172.21.15.175:0/4094270795 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f03a001d7f0 con 0x7f03ac0f51a0 2023-12-06T03:25:48.673 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:48.673 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:48.674 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f039c02ffb0 msgr2=0x7f039c032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:48.674 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f039c02ffb0 0x7f039c032470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f03980096a0 tx=0x7f0398006b40).stop 2023-12-06T03:25:48.674 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 msgr2=0x7f03ac1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:48.674 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac1053b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f03a0013510 tx=0x7f03a000b040).stop 2023-12-06T03:25:48.674 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 shutdown_connections 2023-12-06T03:25:48.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f039c02ffb0 0x7f039c032470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:48.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 --2- 172.21.15.175:0/4094270795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ac0f51a0 0x7f03ac1053b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:48.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 >> 172.21.15.175:0/4094270795 conn(0x7f03ac0f09a0 msgr2=0x7f03ac0f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:48.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 shutdown_connections 2023-12-06T03:25:48.675 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:48.673+0000 7f03b1f1b700 1 -- 172.21.15.175:0/4094270795 wait complete. 2023-12-06T03:25:48.675 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:48.445301+0000 mgr.smithi055.qjkkyb (mgr.14164) 134 : cluster [DBG] pgmap v127: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:48.672520+0000 mon.smithi055 (mon.0) 210 : audit [DBG] from='client.? 172.21.15.175:0/4094270795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:50.064 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:50.065 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.231+0000 7f60c2373700 1 -- 172.21.15.175:0/4291747401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 msgr2=0x7f60bc0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.231+0000 7f60c2373700 1 --2- 172.21.15.175:0/4291747401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc0f5830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f60ac009fb0 tx=0x7f60ac009b30).stop 2023-12-06T03:25:51.233 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.231+0000 7f60c2373700 1 -- 172.21.15.175:0/4291747401 shutdown_connections 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.231+0000 7f60c2373700 1 --2- 172.21.15.175:0/4291747401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc0f5830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.231+0000 7f60c2373700 1 -- 172.21.15.175:0/4291747401 >> 172.21.15.175:0/4291747401 conn(0x7f60bc0f0b70 msgr2=0x7f60bc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.232+0000 7f60c2373700 1 -- 172.21.15.175:0/4291747401 shutdown_connections 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.232+0000 7f60c2373700 1 -- 172.21.15.175:0/4291747401 wait complete. 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.232+0000 7f60c2373700 1 Processor -- start 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.232+0000 7f60c2373700 1 -- start start 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.232+0000 7f60c2373700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:51.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.232+0000 7f60c2373700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc060240 con 0x7f60bc0f5410 2023-12-06T03:25:51.235 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57530/0 (socket says 172.21.15.175:57530) 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60bbfff700 1 -- 172.21.15.175:0/1337058364 learned_addr learned my addr 172.21.15.175:0/1337058364 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60bbfff700 1 -- 172.21.15.175:0/1337058364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac005040 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60bbfff700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc109430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f60ac013510 tx=0x7f60ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60ac009320 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc060440 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60ac013f00 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60ac014d90 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.233+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc109ae0 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.234+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f60ac014420 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.234+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc045ad0 con 0x7f60bc0f5410 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.234+0000 7f60b8ff9700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60a402ff60 0x7f60a4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:51.236 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.234+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f60ac041050 con 0x7f60bc0f5410 2023-12-06T03:25:51.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.234+0000 7f60bb7fe700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60a402ff60 0x7f60a4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:51.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.235+0000 7f60bb7fe700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60a402ff60 0x7f60a4032420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f60b00096a0 tx=0x7f60b0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:51.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.239+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f60ac0095f0 con 0x7f60bc0f5410 2023-12-06T03:25:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.445+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f60bc0fae60 con 0x7f60bc0f5410 2023-12-06T03:25:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.446+0000 7f60b8ff9700 1 -- 172.21.15.175:0/1337058364 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f60ac01db70 con 0x7f60bc0f5410 2023-12-06T03:25:51.447 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:51.447 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60a402ff60 msgr2=0x7f60a4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60a402ff60 0x7f60a4032420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f60b00096a0 tx=0x7f60b0006b40).stop 2023-12-06T03:25:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 msgr2=0x7f60bc109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:51.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc109430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f60ac013510 tx=0x7f60ac00b040).stop 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 shutdown_connections 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f60a402ff60 0x7f60a4032420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 --2- 172.21.15.175:0/1337058364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60bc0f5410 0x7f60bc109430 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 >> 172.21.15.175:0/1337058364 conn(0x7f60bc0f0b70 msgr2=0x7f60bc0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 shutdown_connections 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:51.448+0000 7f60c2373700 1 -- 172.21.15.175:0/1337058364 wait complete. 2023-12-06T03:25:51.450 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:50.445675+0000 mgr.smithi055.qjkkyb (mgr.14164) 135 : cluster [DBG] pgmap v128: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:51.447260+0000 mon.smithi055 (mon.0) 211 : audit [DBG] from='client.? 172.21.15.175:0/1337058364' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:52.834 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:52.835 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:52.445991+0000 mgr.smithi055.qjkkyb (mgr.14164) 136 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 -- 172.21.15.175:0/2096840365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 msgr2=0x7f88200fb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 --2- 172.21.15.175:0/2096840365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88200fb550 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f8810009fb0 tx=0x7f8810009b30).stop 2023-12-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 -- 172.21.15.175:0/2096840365 shutdown_connections 2023-12-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 --2- 172.21.15.175:0/2096840365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88200fb550 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:53.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 -- 172.21.15.175:0/2096840365 >> 172.21.15.175:0/2096840365 conn(0x7f88200f0b70 msgr2=0x7f88200f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 -- 172.21.15.175:0/2096840365 shutdown_connections 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.961+0000 7f8828479700 1 -- 172.21.15.175:0/2096840365 wait complete. 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8828479700 1 Processor -- start 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8828479700 1 -- start start 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8828479700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88201088c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8828479700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820108db0 con 0x7f8820060220 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8826215700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88201088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8826215700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88201088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:57548/0 (socket says 172.21.15.175:57548) 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.962+0000 7f8826215700 1 -- 172.21.15.175:0/1492361460 learned_addr learned my addr 172.21.15.175:0/1492361460 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8826215700 1 -- 172.21.15.175:0/1492361460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8810005040 con 0x7f8820060220 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8826215700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88201088c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f8810013510 tx=0x7f881000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:53.965 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8810009320 con 0x7f8820060220 2023-12-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8820108fb0 con 0x7f8820060220 2023-12-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8810013f00 con 0x7f8820060220 2023-12-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8810014dc0 con 0x7f8820060220 2023-12-06T03:25:53.966 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.963+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8820109440 con 0x7f8820060220 2023-12-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.964+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8820045ad0 con 0x7f8820060220 2023-12-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.964+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f88100095f0 con 0x7f8820060220 2023-12-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.964+0000 7f8816ffd700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f880c02ff60 0x7f880c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.964+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f88100410c0 con 0x7f8820060220 2023-12-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.964+0000 7f8825a14700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f880c02ff60 0x7f880c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:53.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.965+0000 7f8825a14700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f880c02ff60 0x7f880c032420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f881c0096a0 tx=0x7f881c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:53.970 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:53.969+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8810014420 con 0x7f8820060220 2023-12-06T03:25:54.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.176+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f882010a2f0 con 0x7f8820060220 2023-12-06T03:25:54.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.177+0000 7f8816ffd700 1 -- 172.21.15.175:0/1492361460 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8810010230 con 0x7f8820060220 2023-12-06T03:25:54.178 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:54.178 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:54.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f880c02ff60 msgr2=0x7f880c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:54.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f880c02ff60 0x7f880c032420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f881c0096a0 tx=0x7f881c006b40).stop 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 msgr2=0x7f88201088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88201088c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f8810013510 tx=0x7f881000b040).stop 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 shutdown_connections 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f880c02ff60 0x7f880c032420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 --2- 172.21.15.175:0/1492361460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8820060220 0x7f88201088c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 >> 172.21.15.175:0/1492361460 conn(0x7f88200f0b70 msgr2=0x7f88200f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 shutdown_connections 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:54.178+0000 7f8828479700 1 -- 172.21.15.175:0/1492361460 wait complete. 2023-12-06T03:25:54.180 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:54.177746+0000 mon.smithi055 (mon.0) 212 : audit [DBG] from='client.? 172.21.15.175:0/1492361460' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:55.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:54.446360+0000 mgr.smithi055.qjkkyb (mgr.14164) 137 : cluster [DBG] pgmap v130: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:55.572 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:55.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:56.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- 172.21.15.175:0/3587603889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 msgr2=0x7fdfcc0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 --2- 172.21.15.175:0/3587603889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc0f55c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fdfc8009fb0 tx=0x7fdfc8009b30).stop 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- 172.21.15.175:0/3587603889 shutdown_connections 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 --2- 172.21.15.175:0/3587603889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc0f55c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- 172.21.15.175:0/3587603889 >> 172.21.15.175:0/3587603889 conn(0x7fdfcc0f09a0 msgr2=0x7fdfcc0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- 172.21.15.175:0/3587603889 shutdown_connections 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- 172.21.15.175:0/3587603889 wait complete. 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 Processor -- start 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- start start 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc1086c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.697+0000 7fdfd2695700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc108bb0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd1693700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc1086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd1693700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc1086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59178/0 (socket says 172.21.15.175:59178) 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd1693700 1 -- 172.21.15.175:0/2982286473 learned_addr learned my addr 172.21.15.175:0/2982286473 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd1693700 1 -- 172.21.15.175:0/2982286473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8005040 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.700 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd1693700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc1086c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fdfc8013510 tx=0x7fdfc800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:56.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfc8009320 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc108db0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdfc8013f00 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfc8014dc0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.701 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.698+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc109240 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.699+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc045ad0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.699+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fdfc80095f0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.699+0000 7fdfc1ffb700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdfb802ffb0 0x7fdfb8032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:56.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.699+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fdfc8040ff0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.700+0000 7fdfd0e92700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdfb802ffb0 0x7fdfb8032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:56.702 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.700+0000 7fdfd0e92700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdfb802ffb0 0x7fdfb8032470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fdfbc0096a0 tx=0x7fdfbc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:56.706 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.705+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdfc806d2b0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.912 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.911+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdfcc10a0f0 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.913 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.912+0000 7fdfc1ffb700 1 -- 172.21.15.175:0/2982286473 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fdfc8014420 con 0x7fdfcc0f51a0 2023-12-06T03:25:56.914 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:56.914 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:56.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdfb802ffb0 msgr2=0x7fdfb8032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:56.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdfb802ffb0 0x7fdfb8032470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fdfbc0096a0 tx=0x7fdfbc006b40).stop 2023-12-06T03:25:56.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 msgr2=0x7fdfcc1086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:56.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc1086c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fdfc8013510 tx=0x7fdfc800b040).stop 2023-12-06T03:25:56.915 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 shutdown_connections 2023-12-06T03:25:56.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fdfb802ffb0 0x7fdfb8032470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:56.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 --2- 172.21.15.175:0/2982286473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfcc0f51a0 0x7fdfcc1086c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:56.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 >> 172.21.15.175:0/2982286473 conn(0x7fdfcc0f09a0 msgr2=0x7fdfcc05fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:56.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 shutdown_connections 2023-12-06T03:25:56.916 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:56.914+0000 7fdfd2695700 1 -- 172.21.15.175:0/2982286473 wait complete. 2023-12-06T03:25:56.916 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:57.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:56.446731+0000 mgr.smithi055.qjkkyb (mgr.14164) 138 : cluster [DBG] pgmap v131: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:25:57.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:56.913056+0000 mon.smithi055 (mon.0) 213 : audit [DBG] from='client.? 172.21.15.175:0/2982286473' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:25:58.298 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:25:58.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:25:59.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 -- 172.21.15.175:0/988829760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 msgr2=0x7f56800f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:59.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 --2- 172.21.15.175:0/988829760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f56800f6930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f5674009fb0 tx=0x7f5674009b30).stop 2023-12-06T03:25:59.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 -- 172.21.15.175:0/988829760 shutdown_connections 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 --2- 172.21.15.175:0/988829760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f56800f6930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 -- 172.21.15.175:0/988829760 >> 172.21.15.175:0/988829760 conn(0x7f56800f1c70 msgr2=0x7f56800f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 -- 172.21.15.175:0/988829760 shutdown_connections 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.481+0000 7f5684d30700 1 -- 172.21.15.175:0/988829760 wait complete. 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f5684d30700 1 Processor -- start 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f5684d30700 1 -- start start 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f5684d30700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f5680106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f5684d30700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680106c20 con 0x7f56800f6510 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f567e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f5680106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f567e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f5680106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59182/0 (socket says 172.21.15.175:59182) 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f567e59c700 1 -- 172.21.15.175:0/3961643825 learned_addr learned my addr 172.21.15.175:0/3961643825 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f567e59c700 1 -- 172.21.15.175:0/3961643825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674005040 con 0x7f56800f6510 2023-12-06T03:25:59.484 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.482+0000 7f567e59c700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f5680106730 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f5674013510 tx=0x7f567400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:59.485 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.483+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5674009320 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.483+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5674013f00 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.483+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680106e20 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.483+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5674014d90 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.483+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56801072b0 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.483+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f56740263a0 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.484+0000 7f566effd700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.484+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5680045ad0 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.484+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5674041310 con 0x7f56800f6510 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.484+0000 7f567dd9b700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:25:59.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.484+0000 7f567dd9b700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f56700096a0 tx=0x7f5670006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:25:59.490 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.489+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5674012920 con 0x7f56800f6510 2023-12-06T03:25:59.696 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.695+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f56800fc050 con 0x7f56800f6510 2023-12-06T03:25:59.697 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.695+0000 7f566effd700 1 -- 172.21.15.175:0/3961643825 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f56740036f0 con 0x7f56800f6510 2023-12-06T03:25:59.697 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:25:59.697 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:25:59.698 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 msgr2=0x7f5668032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:59.698 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f56700096a0 tx=0x7f5670006b40).stop 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 msgr2=0x7f5680106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f5680106730 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f5674013510 tx=0x7f567400b040).stop 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 shutdown_connections 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f566802ff60 0x7f5668032420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 --2- 172.21.15.175:0/3961643825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56800f6510 0x7f5680106730 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 >> 172.21.15.175:0/3961643825 conn(0x7f56800f1c70 msgr2=0x7f56800f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 shutdown_connections 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:25:59.697+0000 7f5684d30700 1 -- 172.21.15.175:0/3961643825 wait complete. 2023-12-06T03:25:59.699 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:25:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:25:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:25:58.447103+0000 mgr.smithi055.qjkkyb (mgr.14164) 139 : cluster [DBG] pgmap v132: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:00.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:00 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:25:59.696594+0000 mon.smithi055 (mon.0) 214 : audit [DBG] from='client.? 172.21.15.175:0/3961643825' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:01.079 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:01.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:00.447352+0000 mgr.smithi055.qjkkyb (mgr.14164) 140 : cluster [DBG] pgmap v133: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/3249041035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 msgr2=0x7f7fdc0f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 --2- 172.21.15.175:0/3249041035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f8f80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f7fd8009fb0 tx=0x7f7fd8009b30).stop 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/3249041035 shutdown_connections 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 --2- 172.21.15.175:0/3249041035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f8f80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/3249041035 >> 172.21.15.175:0/3249041035 conn(0x7f7fdc0f09a0 msgr2=0x7f7fdc0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/3249041035 shutdown_connections 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/3249041035 wait complete. 2023-12-06T03:26:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.313+0000 7f7fe2b0a700 1 Processor -- start 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe2b0a700 1 -- start start 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe2b0a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe2b0a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0f4e80 con 0x7f7fdc060180 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe1b08700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe1b08700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59194/0 (socket says 172.21.15.175:59194) 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe1b08700 1 -- 172.21.15.175:0/1737836685 learned_addr learned my addr 172.21.15.175:0/1737836685 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe1b08700 1 -- 172.21.15.175:0/1737836685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8005040 con 0x7f7fdc060180 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fe1b08700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f6a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f7fd8013510 tx=0x7f7fd800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.314+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fd8009320 con 0x7f7fdc060180 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7fd8013f00 con 0x7f7fdc060180 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc0f5020 con 0x7f7fdc060180 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fd8014d90 con 0x7f7fdc060180 2023-12-06T03:26:02.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc0f54b0 con 0x7f7fdc060180 2023-12-06T03:26:02.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7fd80095f0 con 0x7f7fdc060180 2023-12-06T03:26:02.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc059100 con 0x7f7fdc060180 2023-12-06T03:26:02.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.315+0000 7f7fd27fc700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7fc802ff60 0x7f7fc8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:02.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.316+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7fd8040fe0 con 0x7f7fdc060180 2023-12-06T03:26:02.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.316+0000 7f7fe1307700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7fc802ff60 0x7f7fc8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:02.319 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.316+0000 7f7fe1307700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7fc802ff60 0x7f7fc8032420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7fcc0096a0 tx=0x7f7fcc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:02.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.320+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7fd801d7f0 con 0x7f7fdc060180 2023-12-06T03:26:02.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.526+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7fdc0fbce0 con 0x7f7fdc060180 2023-12-06T03:26:02.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.526+0000 7f7fd27fc700 1 -- 172.21.15.175:0/1737836685 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7fd8010230 con 0x7f7fdc060180 2023-12-06T03:26:02.527 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:02.528 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:02.529 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7fc802ff60 msgr2=0x7f7fc8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7fc802ff60 0x7f7fc8032420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7fcc0096a0 tx=0x7f7fcc006b40).stop 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 msgr2=0x7f7fdc0f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f6a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f7fd8013510 tx=0x7f7fd800b040).stop 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 shutdown_connections 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7fc802ff60 0x7f7fc8032420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 --2- 172.21.15.175:0/1737836685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7fdc060180 0x7f7fdc0f6a00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 >> 172.21.15.175:0/1737836685 conn(0x7f7fdc0f09a0 msgr2=0x7f7fdc0f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 shutdown_connections 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:02.528+0000 7f7fe2b0a700 1 -- 172.21.15.175:0/1737836685 wait complete. 2023-12-06T03:26:02.530 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:03.923 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:03.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:02.447638+0000 mgr.smithi055.qjkkyb (mgr.14164) 141 : cluster [DBG] pgmap v134: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:02.527454+0000 mon.smithi055 (mon.0) 215 : audit [DBG] from='client.? 172.21.15.175:0/1737836685' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:05.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 -- 172.21.15.175:0/926814175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 msgr2=0x7f46980f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 --2- 172.21.15.175:0/926814175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46980f5830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f4688009fb0 tx=0x7f4688009b30).stop 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 -- 172.21.15.175:0/926814175 shutdown_connections 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 --2- 172.21.15.175:0/926814175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46980f5830 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 -- 172.21.15.175:0/926814175 >> 172.21.15.175:0/926814175 conn(0x7f46980f0b70 msgr2=0x7f46980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 -- 172.21.15.175:0/926814175 shutdown_connections 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 -- 172.21.15.175:0/926814175 wait complete. 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 Processor -- start 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.103+0000 7f46a0516700 1 -- start start 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f46a0516700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46981066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f46a0516700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4698106bd0 con 0x7f46980f5410 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f469e2b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46981066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f469e2b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46981066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59212/0 (socket says 172.21.15.175:59212) 2023-12-06T03:26:05.106 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f469e2b2700 1 -- 172.21.15.175:0/3875323792 learned_addr learned my addr 172.21.15.175:0/3875323792 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f469e2b2700 1 -- 172.21.15.175:0/3875323792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688005040 con 0x7f46980f5410 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f469e2b2700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46981066e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f4688013510 tx=0x7f468800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4688009320 con 0x7f46980f5410 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4688013f00 con 0x7f46980f5410 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4698106dd0 con 0x7f46980f5410 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4688014d90 con 0x7f46980f5410 2023-12-06T03:26:05.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.104+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4698107260 con 0x7f46980f5410 2023-12-06T03:26:05.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.105+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f46880095f0 con 0x7f46980f5410 2023-12-06T03:26:05.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.105+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698045ad0 con 0x7f46980f5410 2023-12-06T03:26:05.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.105+0000 7f468effd700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f468402ff60 0x7f4684032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:05.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.105+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4688041000 con 0x7f46980f5410 2023-12-06T03:26:05.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.106+0000 7f469dab1700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f468402ff60 0x7f4684032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:05.108 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.106+0000 7f469dab1700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f468402ff60 0x7f4684032420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f46940096a0 tx=0x7f4694006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:05.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.111+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4688014420 con 0x7f46980f5410 2023-12-06T03:26:05.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.316+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f46980fae60 con 0x7f46980f5410 2023-12-06T03:26:05.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.317+0000 7f468effd700 1 -- 172.21.15.175:0/3875323792 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4688010230 con 0x7f46980f5410 2023-12-06T03:26:05.319 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:05.319 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f468402ff60 msgr2=0x7f4684032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f468402ff60 0x7f4684032420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f46940096a0 tx=0x7f4694006b40).stop 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 msgr2=0x7f46981066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46981066e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f4688013510 tx=0x7f468800b040).stop 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 shutdown_connections 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f468402ff60 0x7f4684032420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 --2- 172.21.15.175:0/3875323792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46980f5410 0x7f46981066e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:05.320 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 >> 172.21.15.175:0/3875323792 conn(0x7f46980f0b70 msgr2=0x7f469805fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:05.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 shutdown_connections 2023-12-06T03:26:05.321 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:05.319+0000 7f46a0516700 1 -- 172.21.15.175:0/3875323792 wait complete. 2023-12-06T03:26:05.321 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:04.448023+0000 mgr.smithi055.qjkkyb (mgr.14164) 142 : cluster [DBG] pgmap v135: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:06.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:06 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:05.318404+0000 mon.smithi055 (mon.0) 216 : audit [DBG] from='client.? 172.21.15.175:0/3875323792' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:06.699 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:06.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:07.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:06.448408+0000 mgr.smithi055.qjkkyb (mgr.14164) 143 : cluster [DBG] pgmap v136: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 -- 172.21.15.175:0/181935594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 msgr2=0x7f0fd80f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 --2- 172.21.15.175:0/181935594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd80f6930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f0fc8009fb0 tx=0x7f0fc8009b30).stop 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 -- 172.21.15.175:0/181935594 shutdown_connections 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 --2- 172.21.15.175:0/181935594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd80f6930 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 -- 172.21.15.175:0/181935594 >> 172.21.15.175:0/181935594 conn(0x7f0fd80f1c70 msgr2=0x7f0fd80f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 -- 172.21.15.175:0/181935594 shutdown_connections 2023-12-06T03:26:07.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.859+0000 7f0fde1a1700 1 -- 172.21.15.175:0/181935594 wait complete. 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.860+0000 7f0fde1a1700 1 Processor -- start 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.860+0000 7f0fde1a1700 1 -- start start 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fde1a1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd8106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fde1a1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd8106c20 con 0x7f0fd80f6510 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fd77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd8106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fd77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd8106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:60178/0 (socket says 172.21.15.175:60178) 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fd77fe700 1 -- 172.21.15.175:0/737833951 learned_addr learned my addr 172.21.15.175:0/737833951 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fd77fe700 1 -- 172.21.15.175:0/737833951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8005040 con 0x7f0fd80f6510 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fd77fe700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd8106730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f0fc8013510 tx=0x7f0fc800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fc8009320 con 0x7f0fd80f6510 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fc8013f00 con 0x7f0fd80f6510 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd8106e20 con 0x7f0fd80f6510 2023-12-06T03:26:07.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.861+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fc8014d90 con 0x7f0fd80f6510 2023-12-06T03:26:07.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.862+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd81072b0 con 0x7f0fd80f6510 2023-12-06T03:26:07.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.862+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0fc80095f0 con 0x7f0fd80f6510 2023-12-06T03:26:07.864 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.862+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd8045ad0 con 0x7f0fd80f6510 2023-12-06T03:26:07.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.862+0000 7f0fbffff700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0fc002ff60 0x7f0fc0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:07.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.862+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0fc80410c0 con 0x7f0fd80f6510 2023-12-06T03:26:07.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.863+0000 7f0fd6ffd700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0fc002ff60 0x7f0fc0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:07.865 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.863+0000 7f0fd6ffd700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0fc002ff60 0x7f0fc0032420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f0fcc0096a0 tx=0x7f0fcc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:07.868 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:07.867+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0fc8014420 con 0x7f0fd80f6510 2023-12-06T03:26:08.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.073+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0fd80fbf60 con 0x7f0fd80f6510 2023-12-06T03:26:08.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.074+0000 7f0fbffff700 1 -- 172.21.15.175:0/737833951 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0fc801d7f0 con 0x7f0fd80f6510 2023-12-06T03:26:08.075 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:08.075 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:08.076 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0fc002ff60 msgr2=0x7f0fc0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:08.076 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0fc002ff60 0x7f0fc0032420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f0fcc0096a0 tx=0x7f0fcc006b40).stop 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 msgr2=0x7f0fd8106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd8106730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f0fc8013510 tx=0x7f0fc800b040).stop 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 shutdown_connections 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0fc002ff60 0x7f0fc0032420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 --2- 172.21.15.175:0/737833951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fd80f6510 0x7f0fd8106730 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 >> 172.21.15.175:0/737833951 conn(0x7f0fd80f1c70 msgr2=0x7f0fd80f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 shutdown_connections 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:08.075+0000 7f0fde1a1700 1 -- 172.21.15.175:0/737833951 wait complete. 2023-12-06T03:26:08.077 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:08.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:08 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:08.074856+0000 mon.smithi055 (mon.0) 217 : audit [DBG] from='client.? 172.21.15.175:0/737833951' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:09.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:08.448747+0000 mgr.smithi055.qjkkyb (mgr.14164) 144 : cluster [DBG] pgmap v137: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:09.462 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:09.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2224671566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 msgr2=0x7ffae80f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 --2- 172.21.15.175:0/2224671566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80f6e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ffad8009fb0 tx=0x7ffad8009b30).stop 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2224671566 shutdown_connections 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 --2- 172.21.15.175:0/2224671566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80f6e30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2224671566 >> 172.21.15.175:0/2224671566 conn(0x7ffae80f02e0 msgr2=0x7ffae80f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2224671566 shutdown_connections 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2224671566 wait complete. 2023-12-06T03:26:10.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 Processor -- start 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- start start 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.626+0000 7ffaee2c8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae80fb820 con 0x7ffae80f6a10 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:60194/0 (socket says 172.21.15.175:60194) 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae7fff700 1 -- 172.21.15.175:0/2397282971 learned_addr learned my addr 172.21.15.175:0/2397282971 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae7fff700 1 -- 172.21.15.175:0/2397282971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad8005040 con 0x7ffae80f6a10 2023-12-06T03:26:10.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae7fff700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80fd3a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7ffad8009f80 tx=0x7ffad800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffad8009320 con 0x7ffae80f6a10 2023-12-06T03:26:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae80fb9c0 con 0x7ffae80f6a10 2023-12-06T03:26:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffad8013ec0 con 0x7ffae80f6a10 2023-12-06T03:26:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffad801ece0 con 0x7ffae80f6a10 2023-12-06T03:26:10.630 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.627+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae80fbe50 con 0x7ffae80f6a10 2023-12-06T03:26:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.628+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ffad801e420 con 0x7ffae80f6a10 2023-12-06T03:26:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.628+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae8059100 con 0x7ffae80f6a10 2023-12-06T03:26:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.628+0000 7ffae4ff9700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.628+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ffad8040e50 con 0x7ffae80f6a10 2023-12-06T03:26:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.629+0000 7ffae77fe700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:10.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.629+0000 7ffae77fe700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7ffadc0096a0 tx=0x7ffadc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:10.634 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.633+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ffad801e730 con 0x7ffae80f6a10 2023-12-06T03:26:10.842 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.841+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffae8045ad0 con 0x7ffae80f6a10 2023-12-06T03:26:10.843 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.842+0000 7ffae4ff9700 1 -- 172.21.15.175:0/2397282971 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ffad801cb10 con 0x7ffae80f6a10 2023-12-06T03:26:10.843 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:10.843 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:10.844 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 msgr2=0x7ffad0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:10.844 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7ffadc0096a0 tx=0x7ffadc006b40).stop 2023-12-06T03:26:10.844 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 msgr2=0x7ffae80fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80fd3a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7ffad8009f80 tx=0x7ffad800b040).stop 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 shutdown_connections 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ffad002ff60 0x7ffad0032420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 --2- 172.21.15.175:0/2397282971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80f6a10 0x7ffae80fd3a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 >> 172.21.15.175:0/2397282971 conn(0x7ffae80f02e0 msgr2=0x7ffae80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 shutdown_connections 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:10.843+0000 7ffaee2c8700 1 -- 172.21.15.175:0/2397282971 wait complete. 2023-12-06T03:26:10.845 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:10.448986+0000 mgr.smithi055.qjkkyb (mgr.14164) 145 : cluster [DBG] pgmap v138: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:10.842873+0000 mon.smithi055 (mon.0) 218 : audit [DBG] from='client.? 172.21.15.175:0/2397282971' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:12.253 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:12.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:13.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 -- 172.21.15.175:0/1029040921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 msgr2=0x7fa16c0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:13.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 --2- 172.21.15.175:0/1029040921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c0f6930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fa160009fb0 tx=0x7fa160009b30).stop 2023-12-06T03:26:13.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 -- 172.21.15.175:0/1029040921 shutdown_connections 2023-12-06T03:26:13.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 --2- 172.21.15.175:0/1029040921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c0f6930 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 -- 172.21.15.175:0/1029040921 >> 172.21.15.175:0/1029040921 conn(0x7fa16c0f1c70 msgr2=0x7fa16c0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 -- 172.21.15.175:0/1029040921 shutdown_connections 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 -- 172.21.15.175:0/1029040921 wait complete. 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.350+0000 7fa1731c3700 1 Processor -- start 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa1731c3700 1 -- start start 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa1731c3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa1731c3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c069e20 con 0x7fa16c0f6510 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa170f5f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa170f5f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:60214/0 (socket says 172.21.15.175:60214) 2023-12-06T03:26:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa170f5f700 1 -- 172.21.15.175:0/4050266099 learned_addr learned my addr 172.21.15.175:0/4050266099 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.351+0000 7fa170f5f700 1 -- 172.21.15.175:0/4050266099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160005040 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.352+0000 7fa170f5f700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c06b9a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fa160013510 tx=0x7fa16000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.352+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa160009320 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.352+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c069fc0 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.352+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa160013f00 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.352+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa160014d90 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.352+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c06a450 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.353+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa1600095f0 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.353+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c059100 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.353+0000 7fa1697fa700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa15c02ff60 0x7fa15c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.353+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa1600410c0 con 0x7fa16c0f6510 2023-12-06T03:26:13.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.353+0000 7fa16bfff700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa15c02ff60 0x7fa15c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:13.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.353+0000 7fa16bfff700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa15c02ff60 0x7fa15c032420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa1580096a0 tx=0x7fa158006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:13.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.358+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa1600268c0 con 0x7fa16c0f6510 2023-12-06T03:26:13.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.565+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa16c0fbf60 con 0x7fa16c0f6510 2023-12-06T03:26:13.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.565+0000 7fa1697fa700 1 -- 172.21.15.175:0/4050266099 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa160010230 con 0x7fa16c0f6510 2023-12-06T03:26:13.567 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:13.567 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:13.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa15c02ff60 msgr2=0x7fa15c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:13.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa15c02ff60 0x7fa15c032420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fa1580096a0 tx=0x7fa158006b40).stop 2023-12-06T03:26:13.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 msgr2=0x7fa16c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:13.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c06b9a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fa160013510 tx=0x7fa16000b040).stop 2023-12-06T03:26:13.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 shutdown_connections 2023-12-06T03:26:13.568 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa15c02ff60 0x7fa15c032420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:13.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 --2- 172.21.15.175:0/4050266099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa16c0f6510 0x7fa16c06b9a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:13.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 >> 172.21.15.175:0/4050266099 conn(0x7fa16c0f1c70 msgr2=0x7fa16c0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:13.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 shutdown_connections 2023-12-06T03:26:13.569 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:13.567+0000 7fa1731c3700 1 -- 172.21.15.175:0/4050266099 wait complete. 2023-12-06T03:26:13.569 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:12.449359+0000 mgr.smithi055.qjkkyb (mgr.14164) 146 : cluster [DBG] pgmap v139: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:14.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:13.566546+0000 mon.smithi055 (mon.0) 219 : audit [DBG] from='client.? 172.21.15.175:0/4050266099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:14.980 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:14.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:15.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:14.449616+0000 mgr.smithi055.qjkkyb (mgr.14164) 147 : cluster [DBG] pgmap v140: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:16.207 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 -- 172.21.15.175:0/300225713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 msgr2=0x7fb9640f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:16.207 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 --2- 172.21.15.175:0/300225713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb9640f5830 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fb958009fb0 tx=0x7fb958009b30).stop 2023-12-06T03:26:16.207 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 -- 172.21.15.175:0/300225713 shutdown_connections 2023-12-06T03:26:16.207 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 --2- 172.21.15.175:0/300225713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb9640f5830 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:16.207 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 -- 172.21.15.175:0/300225713 >> 172.21.15.175:0/300225713 conn(0x7fb9640f0b70 msgr2=0x7fb9640f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:16.207 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 -- 172.21.15.175:0/300225713 shutdown_connections 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.204+0000 7fb96aca9700 1 -- 172.21.15.175:0/300225713 wait complete. 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.205+0000 7fb96aca9700 1 Processor -- start 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.205+0000 7fb96aca9700 1 -- start start 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.205+0000 7fb96aca9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb964109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.205+0000 7fb96aca9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964060240 con 0x7fb9640f5410 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.205+0000 7fb968a45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb964109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb968a45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb964109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41862/0 (socket says 172.21.15.175:41862) 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb968a45700 1 -- 172.21.15.175:0/99955395 learned_addr learned my addr 172.21.15.175:0/99955395 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:16.208 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb968a45700 1 -- 172.21.15.175:0/99955395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958005040 con 0x7fb9640f5410 2023-12-06T03:26:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb968a45700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb964109430 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb958013510 tx=0x7fb95800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb958009320 con 0x7fb9640f5410 2023-12-06T03:26:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964060440 con 0x7fb9640f5410 2023-12-06T03:26:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb958013f00 con 0x7fb9640f5410 2023-12-06T03:26:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb958014d90 con 0x7fb9640f5410 2023-12-06T03:26:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.206+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb964109ae0 con 0x7fb9640f5410 2023-12-06T03:26:16.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.207+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb9580095f0 con 0x7fb9640f5410 2023-12-06T03:26:16.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.207+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb964045ad0 con 0x7fb9640f5410 2023-12-06T03:26:16.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.207+0000 7fb9617fa700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb95402ffb0 0x7fb954032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:16.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.207+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb958040fe0 con 0x7fb9640f5410 2023-12-06T03:26:16.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.208+0000 7fb963fff700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb95402ffb0 0x7fb954032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:16.212 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.208+0000 7fb963fff700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb95402ffb0 0x7fb954032470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb9500096a0 tx=0x7fb950006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:16.213 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.213+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb95806d2b0 con 0x7fb9640f5410 2023-12-06T03:26:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.420+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb9640fae60 con 0x7fb9640f5410 2023-12-06T03:26:16.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.420+0000 7fb9617fa700 1 -- 172.21.15.175:0/99955395 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb958014420 con 0x7fb9640f5410 2023-12-06T03:26:16.422 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:16.422 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb95402ffb0 msgr2=0x7fb954032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb95402ffb0 0x7fb954032470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb9500096a0 tx=0x7fb950006b40).stop 2023-12-06T03:26:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 msgr2=0x7fb964109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb964109430 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7fb958013510 tx=0x7fb95800b040).stop 2023-12-06T03:26:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 shutdown_connections 2023-12-06T03:26:16.423 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb95402ffb0 0x7fb954032470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:16.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 --2- 172.21.15.175:0/99955395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9640f5410 0x7fb964109430 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:16.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 >> 172.21.15.175:0/99955395 conn(0x7fb9640f0b70 msgr2=0x7fb9640f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:16.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 shutdown_connections 2023-12-06T03:26:16.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:16.422+0000 7fb96aca9700 1 -- 172.21.15.175:0/99955395 wait complete. 2023-12-06T03:26:16.424 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:16.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:16 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:16.421564+0000 mon.smithi055 (mon.0) 220 : audit [DBG] from='client.? 172.21.15.175:0/99955395' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:17.837 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:17.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:16.449871+0000 mgr.smithi055.qjkkyb (mgr.14164) 148 : cluster [DBG] pgmap v141: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- 172.21.15.175:0/2745818692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 msgr2=0x7f23d40f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 --2- 172.21.15.175:0/2745818692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d40f66c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f23d0009fb0 tx=0x7f23d0009b30).stop 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- 172.21.15.175:0/2745818692 shutdown_connections 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 --2- 172.21.15.175:0/2745818692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d40f66c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- 172.21.15.175:0/2745818692 >> 172.21.15.175:0/2745818692 conn(0x7f23d40f1aa0 msgr2=0x7f23d40f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- 172.21.15.175:0/2745818692 shutdown_connections 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- 172.21.15.175:0/2745818692 wait complete. 2023-12-06T03:26:18.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 Processor -- start 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- start start 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d41064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.959+0000 7f23dab36700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41069a0 con 0x7f23d40f62a0 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23d9b34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d41064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23d9b34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d41064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41890/0 (socket says 172.21.15.175:41890) 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23d9b34700 1 -- 172.21.15.175:0/2557628851 learned_addr learned my addr 172.21.15.175:0/2557628851 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23d9b34700 1 -- 172.21.15.175:0/2557628851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0005040 con 0x7f23d40f62a0 2023-12-06T03:26:18.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23d9b34700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d41064b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f23d0013510 tx=0x7f23d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:18.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23d0009320 con 0x7f23d40f62a0 2023-12-06T03:26:18.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4106ba0 con 0x7f23d40f62a0 2023-12-06T03:26:18.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f23d0013f00 con 0x7f23d40f62a0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23d0014dc0 con 0x7f23d40f62a0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.960+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d4107030 con 0x7f23d40f62a0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.961+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f23d00095f0 con 0x7f23d40f62a0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.961+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d4045ad0 con 0x7f23d40f62a0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.961+0000 7f23ca7fc700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f23c002ff60 0x7f23c0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.961+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f23d0041040 con 0x7f23d40f62a0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.961+0000 7f23d9333700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f23c002ff60 0x7f23c0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:18.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.962+0000 7f23d9333700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f23c002ff60 0x7f23c0032420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f23c40096a0 tx=0x7f23c4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:18.967 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:18.966+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f23d0014420 con 0x7f23d40f62a0 2023-12-06T03:26:19.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.172+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f23d4107ee0 con 0x7f23d40f62a0 2023-12-06T03:26:19.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.172+0000 7f23ca7fc700 1 -- 172.21.15.175:0/2557628851 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f23d0010230 con 0x7f23d40f62a0 2023-12-06T03:26:19.174 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:19.175 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:19.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f23c002ff60 msgr2=0x7f23c0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:19.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f23c002ff60 0x7f23c0032420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f23c40096a0 tx=0x7f23c4006b40).stop 2023-12-06T03:26:19.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 msgr2=0x7f23d41064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d41064b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f23d0013510 tx=0x7f23d000b040).stop 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 shutdown_connections 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f23c002ff60 0x7f23c0032420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 --2- 172.21.15.175:0/2557628851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40f62a0 0x7f23d41064b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 >> 172.21.15.175:0/2557628851 conn(0x7f23d40f1aa0 msgr2=0x7f23d40f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 shutdown_connections 2023-12-06T03:26:19.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:19.174+0000 7f23dab36700 1 -- 172.21.15.175:0/2557628851 wait complete. 2023-12-06T03:26:19.178 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:19.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:18.450243+0000 mgr.smithi055.qjkkyb (mgr.14164) 149 : cluster [DBG] pgmap v142: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:19.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:19.173413+0000 mon.smithi055 (mon.0) 221 : audit [DBG] from='client.? 172.21.15.175:0/2557628851' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:20.578 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:20.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:21.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 -- 172.21.15.175:0/2490656814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 msgr2=0x7f6ca00f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:21.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 --2- 172.21.15.175:0/2490656814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca00f90f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6c90009fb0 tx=0x7f6c90009b30).stop 2023-12-06T03:26:21.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 -- 172.21.15.175:0/2490656814 shutdown_connections 2023-12-06T03:26:21.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 --2- 172.21.15.175:0/2490656814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca00f90f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:21.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 -- 172.21.15.175:0/2490656814 >> 172.21.15.175:0/2490656814 conn(0x7f6ca00f0b70 msgr2=0x7f6ca00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:21.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 -- 172.21.15.175:0/2490656814 shutdown_connections 2023-12-06T03:26:21.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.785+0000 7f6ca6553700 1 -- 172.21.15.175:0/2490656814 wait complete. 2023-12-06T03:26:21.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6ca6553700 1 Processor -- start 2023-12-06T03:26:21.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6ca6553700 1 -- start start 2023-12-06T03:26:21.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6ca6553700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca01066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:21.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6ca6553700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca0106bd0 con 0x7f6ca0060280 2023-12-06T03:26:21.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca01066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6c9ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca01066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41910/0 (socket says 172.21.15.175:41910) 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.786+0000 7f6c9ffff700 1 -- 172.21.15.175:0/1455636732 learned_addr learned my addr 172.21.15.175:0/1455636732 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6c9ffff700 1 -- 172.21.15.175:0/1455636732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90005040 con 0x7f6ca0060280 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6c9ffff700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca01066e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f6c90013510 tx=0x7f6c9000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c90009320 con 0x7f6ca0060280 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca0106dd0 con 0x7f6ca0060280 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c90013f00 con 0x7f6ca0060280 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c90014dc0 con 0x7f6ca0060280 2023-12-06T03:26:21.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.787+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca0107260 con 0x7f6ca0060280 2023-12-06T03:26:21.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.788+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6c900095f0 con 0x7f6ca0060280 2023-12-06T03:26:21.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.788+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca0045ad0 con 0x7f6ca0060280 2023-12-06T03:26:21.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.788+0000 7f6c9cff9700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6c8802ff60 0x7f6c88032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:21.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.788+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6c90041000 con 0x7f6ca0060280 2023-12-06T03:26:21.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.788+0000 7f6c9f7fe700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6c8802ff60 0x7f6c88032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:21.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.789+0000 7f6c9f7fe700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6c8802ff60 0x7f6c88032420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f6c940096a0 tx=0x7f6c94006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:21.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.793+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6c9001c790 con 0x7f6ca0060280 2023-12-06T03:26:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:20.450610+0000 mgr.smithi055.qjkkyb (mgr.14164) 150 : cluster [DBG] pgmap v143: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:21.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.996+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6ca0108110 con 0x7f6ca0060280 2023-12-06T03:26:21.998 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.997+0000 7f6c9cff9700 1 -- 172.21.15.175:0/1455636732 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6c90010230 con 0x7f6ca0060280 2023-12-06T03:26:21.998 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:21.998 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:21.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6c8802ff60 msgr2=0x7f6c88032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:21.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6c8802ff60 0x7f6c88032420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f6c940096a0 tx=0x7f6c94006b40).stop 2023-12-06T03:26:21.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 msgr2=0x7f6ca01066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:21.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca01066e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f6c90013510 tx=0x7f6c9000b040).stop 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 shutdown_connections 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6c8802ff60 0x7f6c88032420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 --2- 172.21.15.175:0/1455636732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6ca0060280 0x7f6ca01066e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 >> 172.21.15.175:0/1455636732 conn(0x7f6ca00f0b70 msgr2=0x7f6ca00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 shutdown_connections 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:21.998+0000 7f6ca6553700 1 -- 172.21.15.175:0/1455636732 wait complete. 2023-12-06T03:26:22.000 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:22.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:21.997765+0000 mon.smithi055 (mon.0) 222 : audit [DBG] from='client.? 172.21.15.175:0/1455636732' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:23.373 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:23.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:22.450992+0000 mgr.smithi055.qjkkyb (mgr.14164) 151 : cluster [DBG] pgmap v144: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:22.553605+0000 mon.smithi055 (mon.0) 223 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:26:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:22.556600+0000 mon.smithi055 (mon.0) 224 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:26:24.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 -- 172.21.15.175:0/1672981097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 msgr2=0x7fb2e40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:24.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 --2- 172.21.15.175:0/1672981097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e40f5830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fb2d4009fb0 tx=0x7fb2d4009b30).stop 2023-12-06T03:26:24.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 -- 172.21.15.175:0/1672981097 shutdown_connections 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 --2- 172.21.15.175:0/1672981097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e40f5830 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 -- 172.21.15.175:0/1672981097 >> 172.21.15.175:0/1672981097 conn(0x7fb2e40f0b70 msgr2=0x7fb2e40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 -- 172.21.15.175:0/1672981097 shutdown_connections 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.515+0000 7fb2ec482700 1 -- 172.21.15.175:0/1672981097 wait complete. 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ec482700 1 Processor -- start 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ec482700 1 -- start start 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ec482700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e4109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ec482700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4060220 con 0x7fb2e40f5410 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ea21e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e4109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ea21e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e4109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41922/0 (socket says 172.21.15.175:41922) 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ea21e700 1 -- 172.21.15.175:0/2199962215 learned_addr learned my addr 172.21.15.175:0/2199962215 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ea21e700 1 -- 172.21.15.175:0/2199962215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4005040 con 0x7fb2e40f5410 2023-12-06T03:26:24.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.516+0000 7fb2ea21e700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e4109450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fb2d4013510 tx=0x7fb2d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:24.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.517+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2d4009320 con 0x7fb2e40f5410 2023-12-06T03:26:24.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.517+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e4060420 con 0x7fb2e40f5410 2023-12-06T03:26:24.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.517+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2d4013f00 con 0x7fb2e40f5410 2023-12-06T03:26:24.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.517+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2d4014d90 con 0x7fb2e40f5410 2023-12-06T03:26:24.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.517+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4109b00 con 0x7fb2e40f5410 2023-12-06T03:26:24.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.517+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb2d40263a0 con 0x7fb2e40f5410 2023-12-06T03:26:24.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.518+0000 7fb2daffd700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb2d002ff60 0x7fb2d0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:24.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.518+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e4059100 con 0x7fb2e40f5410 2023-12-06T03:26:24.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.518+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb2d40410f0 con 0x7fb2e40f5410 2023-12-06T03:26:24.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.518+0000 7fb2e9a1d700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb2d002ff60 0x7fb2d0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:24.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.518+0000 7fb2e9a1d700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb2d002ff60 0x7fb2d0032420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb2e00096a0 tx=0x7fb2e0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:24.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.523+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb2d4012920 con 0x7fb2e40f5410 2023-12-06T03:26:24.730 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.729+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb2e4045ad0 con 0x7fb2e40f5410 2023-12-06T03:26:24.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.730+0000 7fb2daffd700 1 -- 172.21.15.175:0/2199962215 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb2d402b020 con 0x7fb2e40f5410 2023-12-06T03:26:24.732 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:24.732 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:24.732 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb2d002ff60 msgr2=0x7fb2d0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb2d002ff60 0x7fb2d0032420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb2e00096a0 tx=0x7fb2e0006b40).stop 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 msgr2=0x7fb2e4109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e4109450 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fb2d4013510 tx=0x7fb2d400b040).stop 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 shutdown_connections 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb2d002ff60 0x7fb2d0032420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 --2- 172.21.15.175:0/2199962215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2e40f5410 0x7fb2e4109450 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 >> 172.21.15.175:0/2199962215 conn(0x7fb2e40f0b70 msgr2=0x7fb2e40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 shutdown_connections 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:24.731+0000 7fb2ec482700 1 -- 172.21.15.175:0/2199962215 wait complete. 2023-12-06T03:26:24.733 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:25.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:24.451372+0000 mgr.smithi055.qjkkyb (mgr.14164) 152 : cluster [DBG] pgmap v145: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:25.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:24.730930+0000 mon.smithi055 (mon.0) 225 : audit [DBG] from='client.? 172.21.15.175:0/2199962215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:26.117 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:26.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 -- 172.21.15.175:0/76901681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 msgr2=0x7f65a80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 --2- 172.21.15.175:0/76901681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a80f55c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f6598009fb0 tx=0x7f6598009b30).stop 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 -- 172.21.15.175:0/76901681 shutdown_connections 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 --2- 172.21.15.175:0/76901681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a80f55c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 -- 172.21.15.175:0/76901681 >> 172.21.15.175:0/76901681 conn(0x7f65a80f09a0 msgr2=0x7f65a80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 -- 172.21.15.175:0/76901681 shutdown_connections 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 -- 172.21.15.175:0/76901681 wait complete. 2023-12-06T03:26:27.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 Processor -- start 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.309+0000 7f65a759e700 1 -- start start 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a8109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8060280 con 0x7f65a80f51a0 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a8109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a8109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49858/0 (socket says 172.21.15.175:49858) 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a659c700 1 -- 172.21.15.175:0/3492829504 learned_addr learned my addr 172.21.15.175:0/3492829504 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a659c700 1 -- 172.21.15.175:0/3492829504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598005040 con 0x7f65a80f51a0 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a659c700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a8109170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f6598013510 tx=0x7f659800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6598009320 con 0x7f65a80f51a0 2023-12-06T03:26:27.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a8060480 con 0x7f65a80f51a0 2023-12-06T03:26:27.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6598013f00 con 0x7f65a80f51a0 2023-12-06T03:26:27.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6598014dc0 con 0x7f65a80f51a0 2023-12-06T03:26:27.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.310+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a81098a0 con 0x7f65a80f51a0 2023-12-06T03:26:27.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.311+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f65980095f0 con 0x7f65a80f51a0 2023-12-06T03:26:27.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.311+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a8045ad0 con 0x7f65a80f51a0 2023-12-06T03:26:27.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.311+0000 7f6596ffd700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f659002ff60 0x7f6590032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:27.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.311+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6598041080 con 0x7f65a80f51a0 2023-12-06T03:26:27.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.312+0000 7f65a5d9b700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f659002ff60 0x7f6590032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:27.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.312+0000 7f65a5d9b700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f659002ff60 0x7f6590032420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f659c0096a0 tx=0x7f659c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:27.317 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.316+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f659801c790 con 0x7f65a80f51a0 2023-12-06T03:26:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.524+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f65a80fabe0 con 0x7f65a80f51a0 2023-12-06T03:26:27.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.525+0000 7f6596ffd700 1 -- 172.21.15.175:0/3492829504 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6598010230 con 0x7f65a80f51a0 2023-12-06T03:26:27.526 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:27.526 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f659002ff60 msgr2=0x7f6590032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f659002ff60 0x7f6590032420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f659c0096a0 tx=0x7f659c006b40).stop 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 msgr2=0x7f65a8109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a8109170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f6598013510 tx=0x7f659800b040).stop 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 shutdown_connections 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f659002ff60 0x7f6590032420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 --2- 172.21.15.175:0/3492829504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65a80f51a0 0x7f65a8109170 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 >> 172.21.15.175:0/3492829504 conn(0x7f65a80f09a0 msgr2=0x7f65a80f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 shutdown_connections 2023-12-06T03:26:27.528 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:27.526+0000 7f65a759e700 1 -- 172.21.15.175:0/3492829504 wait complete. 2023-12-06T03:26:27.529 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:26.451736+0000 mgr.smithi055.qjkkyb (mgr.14164) 153 : cluster [DBG] pgmap v146: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:28.919 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:28.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:28.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:27.526058+0000 mon.smithi055 (mon.0) 226 : audit [DBG] from='client.? 172.21.15.175:0/3492829504' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:28.452101+0000 mgr.smithi055.qjkkyb (mgr.14164) 154 : cluster [DBG] pgmap v147: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:30.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/682150678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 msgr2=0x7f0abc0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:30.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 --2- 172.21.15.175:0/682150678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc0f6930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f0aac009fb0 tx=0x7f0aac009b30).stop 2023-12-06T03:26:30.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/682150678 shutdown_connections 2023-12-06T03:26:30.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 --2- 172.21.15.175:0/682150678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc0f6930 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:30.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/682150678 >> 172.21.15.175:0/682150678 conn(0x7f0abc0f1c70 msgr2=0x7f0abc0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:30.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/682150678 shutdown_connections 2023-12-06T03:26:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.096+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/682150678 wait complete. 2023-12-06T03:26:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.097+0000 7f0ac3ea7700 1 Processor -- start 2023-12-06T03:26:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.097+0000 7f0ac3ea7700 1 -- start start 2023-12-06T03:26:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.097+0000 7f0ac3ea7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:30.099 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.097+0000 7f0ac3ea7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc106c20 con 0x7f0abc0f6510 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.097+0000 7f0ac1c43700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ac1c43700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49874/0 (socket says 172.21.15.175:49874) 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ac1c43700 1 -- 172.21.15.175:0/3272962267 learned_addr learned my addr 172.21.15.175:0/3272962267 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ac1c43700 1 -- 172.21.15.175:0/3272962267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac005040 con 0x7f0abc0f6510 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ac1c43700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc106730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0aac013510 tx=0x7f0aac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aac009320 con 0x7f0abc0f6510 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc106e20 con 0x7f0abc0f6510 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0aac013f00 con 0x7f0abc0f6510 2023-12-06T03:26:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0aac014d90 con 0x7f0abc0f6510 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.098+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc1072b0 con 0x7f0abc0f6510 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.099+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0aac0095f0 con 0x7f0abc0f6510 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.099+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0abc045ad0 con 0x7f0abc0f6510 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.099+0000 7f0ab27fc700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0aa802ffb0 0x7f0aa8032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.099+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0aac040f90 con 0x7f0abc0f6510 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.099+0000 7f0ac1442700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0aa802ffb0 0x7f0aa8032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:30.102 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.100+0000 7f0ac1442700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0aa802ffb0 0x7f0aa8032470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f0ab80096a0 tx=0x7f0ab8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:30.105 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.104+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0aac014420 con 0x7f0abc0f6510 2023-12-06T03:26:30.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.308+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0abc0fbf60 con 0x7f0abc0f6510 2023-12-06T03:26:30.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.309+0000 7f0ab27fc700 1 -- 172.21.15.175:0/3272962267 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0aac010230 con 0x7f0abc0f6510 2023-12-06T03:26:30.313 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:30.314 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:30.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.310+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0aa802ffb0 msgr2=0x7f0aa8032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:30.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.310+0000 7f0ac3ea7700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0aa802ffb0 0x7f0aa8032470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f0ab80096a0 tx=0x7f0ab8006b40).stop 2023-12-06T03:26:30.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 msgr2=0x7f0abc106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:30.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc106730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0aac013510 tx=0x7f0aac00b040).stop 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 shutdown_connections 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0aa802ffb0 0x7f0aa8032470 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 --2- 172.21.15.175:0/3272962267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0abc0f6510 0x7f0abc106730 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 >> 172.21.15.175:0/3272962267 conn(0x7f0abc0f1c70 msgr2=0x7f0abc0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 shutdown_connections 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:30.311+0000 7f0ac3ea7700 1 -- 172.21.15.175:0/3272962267 wait complete. 2023-12-06T03:26:30.315 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:30.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:30 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:30.310183+0000 mon.smithi055 (mon.0) 227 : audit [DBG] from='client.? 172.21.15.175:0/3272962267' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:31.701 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:31.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:30.452362+0000 mgr.smithi055.qjkkyb (mgr.14164) 155 : cluster [DBG] pgmap v148: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:32.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:32 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:32.339426+0000 mon.smithi055 (mon.0) 228 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi055", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2023-12-06T03:26:32.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:32 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:32.342259+0000 mon.smithi055 (mon.0) 229 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi055", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2023-12-06T03:26:32.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:32 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:32.343085+0000 mon.smithi055 (mon.0) 230 : audit [DBG] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-06T03:26:32.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 -- 172.21.15.175:0/2871680751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 msgr2=0x7f0b300f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:32.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 --2- 172.21.15.175:0/2871680751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b300f6930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0b24009fb0 tx=0x7f0b24009b30).stop 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 -- 172.21.15.175:0/2871680751 shutdown_connections 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 --2- 172.21.15.175:0/2871680751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b300f6930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 -- 172.21.15.175:0/2871680751 >> 172.21.15.175:0/2871680751 conn(0x7f0b300f1c70 msgr2=0x7f0b300f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 -- 172.21.15.175:0/2871680751 shutdown_connections 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.799+0000 7f0b37314700 1 -- 172.21.15.175:0/2871680751 wait complete. 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b37314700 1 Processor -- start 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b37314700 1 -- start start 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b37314700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b30106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b37314700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30106c20 con 0x7f0b300f6510 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b350b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b30106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b350b0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b30106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:49890/0 (socket says 172.21.15.175:49890) 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.800+0000 7f0b350b0700 1 -- 172.21.15.175:0/3413252055 learned_addr learned my addr 172.21.15.175:0/3413252055 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b350b0700 1 -- 172.21.15.175:0/3413252055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b24005040 con 0x7f0b300f6510 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b350b0700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b30106730 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f0b24014000 tx=0x7f0b2400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:32.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b240095f0 con 0x7f0b300f6510 2023-12-06T03:26:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b30106e20 con 0x7f0b300f6510 2023-12-06T03:26:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b24007e70 con 0x7f0b300f6510 2023-12-06T03:26:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b24014ea0 con 0x7f0b300f6510 2023-12-06T03:26:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.801+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b301072b0 con 0x7f0b300f6510 2023-12-06T03:26:32.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.802+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0b24025070 con 0x7f0b300f6510 2023-12-06T03:26:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.802+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30045ad0 con 0x7f0b300f6510 2023-12-06T03:26:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.802+0000 7f0b21ffb700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b1c02ff60 0x7f0b1c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.802+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0b24040940 con 0x7f0b300f6510 2023-12-06T03:26:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.802+0000 7f0b348af700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b1c02ff60 0x7f0b1c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:32.805 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.803+0000 7f0b348af700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b1c02ff60 0x7f0b1c032420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f0b2c0096a0 tx=0x7f0b2c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:32.808 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:32.807+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0b240079c0 con 0x7f0b300f6510 2023-12-06T03:26:33.015 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.013+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b300fbf60 con 0x7f0b300f6510 2023-12-06T03:26:33.015 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.013+0000 7f0b21ffb700 1 -- 172.21.15.175:0/3413252055 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0b2400f300 con 0x7f0b300f6510 2023-12-06T03:26:33.015 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:33.015 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:33.016 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b1c02ff60 msgr2=0x7f0b1c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:33.016 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b1c02ff60 0x7f0b1c032420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f0b2c0096a0 tx=0x7f0b2c006b40).stop 2023-12-06T03:26:33.016 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 msgr2=0x7f0b30106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b30106730 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f0b24014000 tx=0x7f0b2400b040).stop 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 shutdown_connections 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0b1c02ff60 0x7f0b1c032420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 --2- 172.21.15.175:0/3413252055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300f6510 0x7f0b30106730 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 >> 172.21.15.175:0/3413252055 conn(0x7f0b300f1c70 msgr2=0x7f0b300f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 shutdown_connections 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:33.015+0000 7f0b37314700 1 -- 172.21.15.175:0/3413252055 wait complete. 2023-12-06T03:26:33.017 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:33.692 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:26:32.338013+0000 mgr.smithi055.qjkkyb (mgr.14164) 156 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Deploy daemon alertmanager.smithi055 ... 2023-12-06T03:26:33.692 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager 2023-12-06T03:26:33.692 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2023-12-06T03:26:33.693 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:22:11Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.693 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:22:48Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.693 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:23:25Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.693 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2023-12-06T03:26:33.693 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:24:39Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:25:17Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:25:54Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Traceback (most recent call last): 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8029, in 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: main() 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8017, in main 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: r = ctx.func(ctx) 2023-12-06T03:26:33.694 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 1760, in _default_image 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: return func(ctx) 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 4263, in command_deploy 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 4191, in extract_uid_gid_monitoring 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: uid, gid = extract_uid_gid(ctx, file_path=['/etc/alertmanager', '/etc/prometheus']) 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 2531, in extract_uid_gid 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: raise RuntimeError('uid/gid not found') 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: RuntimeError: uid/gid not found 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Traceback (most recent call last): 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1144, in _remote_connection 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: yield (conn, connr) 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1059, in _run_cephadm 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: code, '\n'.join(err))) 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Deploy daemon alertmanager.smithi055 ... 2023-12-06T03:26:33.695 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:22:11Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:22:48Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:23:25Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi055 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0... 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:24:39Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.696 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:25:17Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr time="2023-12-06T03:25:54Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Traceback (most recent call last): 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8029, in 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: main() 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8017, in main 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: r = ctx.func(ctx) 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 1760, in _default_image 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: return func(ctx) 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 4263, in command_deploy 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type) 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 4191, in extract_uid_gid_monitoring 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: uid, gid = extract_uid_gid(ctx, file_path=['/etc/alertmanager', '/etc/prometheus']) 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 2531, in extract_uid_gid 2023-12-06T03:26:33.697 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: raise RuntimeError('uid/gid not found') 2023-12-06T03:26:33.698 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: RuntimeError: uid/gid not found 2023-12-06T03:26:33.698 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:26:32.343839+0000 mgr.smithi055.qjkkyb (mgr.14164) 157 : cephadm [INF] Deploying daemon crash.smithi055 on smithi055 2023-12-06T03:26:33.698 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:32.452603+0000 mgr.smithi055.qjkkyb (mgr.14164) 158 : cluster [DBG] pgmap v149: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:33.698 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:33.014690+0000 mon.smithi055 (mon.0) 231 : audit [DBG] from='client.? 172.21.15.175:0/3413252055' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:34.418 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:34.418 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:34.114763+0000 mon.smithi055 (mon.0) 232 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:34.210205+0000 mon.smithi055 (mon.0) 233 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:34.215020+0000 mon.smithi055 (mon.0) 234 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:34.215546+0000 mon.smithi055 (mon.0) 235 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:34.215992+0000 mgr.smithi055.qjkkyb (mgr.14164) 159 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:34.218799+0000 mon.smithi055 (mon.0) 236 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:26:34.222385+0000 mgr.smithi055.qjkkyb (mgr.14164) 160 : cephadm [INF] Deploying daemon grafana.smithi055 on smithi055 2023-12-06T03:26:35.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:34.452863+0000 mgr.smithi055.qjkkyb (mgr.14164) 161 : cluster [DBG] pgmap v150: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:35.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 -- 172.21.15.175:0/91620136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 msgr2=0x7f55680f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:35.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 --2- 172.21.15.175:0/91620136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f55680f6930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5558009fb0 tx=0x7f5558009b30).stop 2023-12-06T03:26:35.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 -- 172.21.15.175:0/91620136 shutdown_connections 2023-12-06T03:26:35.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 --2- 172.21.15.175:0/91620136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f55680f6930 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:35.590 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 -- 172.21.15.175:0/91620136 >> 172.21.15.175:0/91620136 conn(0x7f55680f1c70 msgr2=0x7f55680f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 -- 172.21.15.175:0/91620136 shutdown_connections 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.587+0000 7f556c99c700 1 -- 172.21.15.175:0/91620136 wait complete. 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556c99c700 1 Processor -- start 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556c99c700 1 -- start start 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f556806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556c99c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5568069e20 con 0x7f55680f6510 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f556806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f556806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:44098/0 (socket says 172.21.15.175:44098) 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.588+0000 7f556659c700 1 -- 172.21.15.175:0/1296663842 learned_addr learned my addr 172.21.15.175:0/1296663842 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f556659c700 1 -- 172.21.15.175:0/1296663842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558005040 con 0x7f55680f6510 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f556659c700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f556806b9a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5558013510 tx=0x7f555800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5558009320 con 0x7f55680f6510 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5568069fc0 con 0x7f55680f6510 2023-12-06T03:26:35.591 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5558013f00 con 0x7f55680f6510 2023-12-06T03:26:35.592 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5558014d90 con 0x7f55680f6510 2023-12-06T03:26:35.592 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.589+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556806a450 con 0x7f55680f6510 2023-12-06T03:26:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.590+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f55580095f0 con 0x7f55680f6510 2023-12-06T03:26:35.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.590+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5568059100 con 0x7f55680f6510 2023-12-06T03:26:35.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.590+0000 7f555effd700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f555402ff60 0x7f5554032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:35.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.590+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5558041000 con 0x7f55680f6510 2023-12-06T03:26:35.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.590+0000 7f5565d9b700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f555402ff60 0x7f5554032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:35.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.590+0000 7f5565d9b700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f555402ff60 0x7f5554032420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f55500096a0 tx=0x7f5550006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:35.596 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.596+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5558014420 con 0x7f55680f6510 2023-12-06T03:26:35.801 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.800+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f55680fbf60 con 0x7f55680f6510 2023-12-06T03:26:35.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f555effd700 1 -- 172.21.15.175:0/1296663842 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5558010230 con 0x7f55680f6510 2023-12-06T03:26:35.802 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:35.802 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:35.803 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f555402ff60 msgr2=0x7f5554032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f555402ff60 0x7f5554032420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f55500096a0 tx=0x7f5550006b40).stop 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 msgr2=0x7f556806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f556806b9a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5558013510 tx=0x7f555800b040).stop 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 shutdown_connections 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f555402ff60 0x7f5554032420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 --2- 172.21.15.175:0/1296663842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55680f6510 0x7f556806b9a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 >> 172.21.15.175:0/1296663842 conn(0x7f55680f1c70 msgr2=0x7f55680f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 shutdown_connections 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:35.802+0000 7f556c99c700 1 -- 172.21.15.175:0/1296663842 wait complete. 2023-12-06T03:26:35.804 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:36.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:35.801928+0000 mon.smithi055 (mon.0) 237 : audit [DBG] from='client.? 172.21.15.175:0/1296663842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:37.214 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:37.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:37.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:36.453145+0000 mgr.smithi055.qjkkyb (mgr.14164) 162 : cluster [DBG] pgmap v151: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 -- 172.21.15.175:0/3077794199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 msgr2=0x7fc33c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 --2- 172.21.15.175:0/3077794199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c0f5830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fc324009fb0 tx=0x7fc324009b30).stop 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 -- 172.21.15.175:0/3077794199 shutdown_connections 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 --2- 172.21.15.175:0/3077794199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c0f5830 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 -- 172.21.15.175:0/3077794199 >> 172.21.15.175:0/3077794199 conn(0x7fc33c0f0b70 msgr2=0x7fc33c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 -- 172.21.15.175:0/3077794199 shutdown_connections 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 -- 172.21.15.175:0/3077794199 wait complete. 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.440+0000 7fc340b16700 1 Processor -- start 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc340b16700 1 -- start start 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc340b16700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc340b16700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c060220 con 0x7fc33c0f5410 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc33a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc33a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:44114/0 (socket says 172.21.15.175:44114) 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc33a59c700 1 -- 172.21.15.175:0/3562451451 learned_addr learned my addr 172.21.15.175:0/3562451451 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc33a59c700 1 -- 172.21.15.175:0/3562451451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc324005040 con 0x7fc33c0f5410 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc33a59c700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c109450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fc324013510 tx=0x7fc32400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc324009320 con 0x7fc33c0f5410 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c060420 con 0x7fc33c0f5410 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc324013f00 con 0x7fc33c0f5410 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.441+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc324014d90 con 0x7fc33c0f5410 2023-12-06T03:26:38.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.442+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c109b00 con 0x7fc33c0f5410 2023-12-06T03:26:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.442+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c059100 con 0x7fc33c0f5410 2023-12-06T03:26:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.442+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fc3240095f0 con 0x7fc33c0f5410 2023-12-06T03:26:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.442+0000 7fc332ffd700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc32802ff60 0x7fc328032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.442+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fc324040fe0 con 0x7fc33c0f5410 2023-12-06T03:26:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.443+0000 7fc339d9b700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc32802ff60 0x7fc328032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.443+0000 7fc339d9b700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc32802ff60 0x7fc328032420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc32c0096a0 tx=0x7fc32c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:38.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.447+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc32401d7f0 con 0x7fc33c0f5410 2023-12-06T03:26:38.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.651+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc33c045ad0 con 0x7fc33c0f5410 2023-12-06T03:26:38.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.652+0000 7fc332ffd700 1 -- 172.21.15.175:0/3562451451 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc324010230 con 0x7fc33c0f5410 2023-12-06T03:26:38.652 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:38.652 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:38.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc32802ff60 msgr2=0x7fc328032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc32802ff60 0x7fc328032420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc32c0096a0 tx=0x7fc32c006b40).stop 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 msgr2=0x7fc33c109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c109450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fc324013510 tx=0x7fc32400b040).stop 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 shutdown_connections 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc32802ff60 0x7fc328032420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 --2- 172.21.15.175:0/3562451451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc33c0f5410 0x7fc33c109450 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 >> 172.21.15.175:0/3562451451 conn(0x7fc33c0f0b70 msgr2=0x7fc33c0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 shutdown_connections 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:38.653+0000 7fc340b16700 1 -- 172.21.15.175:0/3562451451 wait complete. 2023-12-06T03:26:38.654 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:38.453388+0000 mgr.smithi055.qjkkyb (mgr.14164) 163 : cluster [DBG] pgmap v152: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:38.651974+0000 mon.smithi055 (mon.0) 238 : audit [DBG] from='client.? 172.21.15.175:0/3562451451' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:40.037 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:40.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:41.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.190+0000 7f0ca3506700 1 -- 172.21.15.175:0/2104765523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 msgr2=0x7f0c9c0f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:41.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.190+0000 7f0ca3506700 1 --2- 172.21.15.175:0/2104765523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c0f4f20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f0c8c009fb0 tx=0x7f0c8c009b30).stop 2023-12-06T03:26:41.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.190+0000 7f0ca3506700 1 -- 172.21.15.175:0/2104765523 shutdown_connections 2023-12-06T03:26:41.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.190+0000 7f0ca3506700 1 --2- 172.21.15.175:0/2104765523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c0f4f20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:41.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.190+0000 7f0ca3506700 1 -- 172.21.15.175:0/2104765523 >> 172.21.15.175:0/2104765523 conn(0x7f0c9c0f0260 msgr2=0x7f0c9c0f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca3506700 1 -- 172.21.15.175:0/2104765523 shutdown_connections 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca3506700 1 -- 172.21.15.175:0/2104765523 wait complete. 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca3506700 1 Processor -- start 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca3506700 1 -- start start 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca3506700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c106710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca3506700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c106c00 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca12a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca12a2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:44128/0 (socket says 172.21.15.175:44128) 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.191+0000 7f0ca12a2700 1 -- 172.21.15.175:0/1755208797 learned_addr learned my addr 172.21.15.175:0/1755208797 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0ca12a2700 1 -- 172.21.15.175:0/1755208797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c005040 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0ca12a2700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c106710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f0c8c013510 tx=0x7f0c8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c8c009320 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c106e00 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c107290 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c8c013f00 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.192+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c8c014d60 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.193+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0c8c014420 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.193+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c045ad0 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.193+0000 7f0c91ffb700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0c8802ff50 0x7f0c88032410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.193+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0c8c0412d0 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.193+0000 7f0ca0aa1700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0c8802ff50 0x7f0c88032410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:41.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.194+0000 7f0ca0aa1700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0c8802ff50 0x7f0c88032410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0c980096a0 tx=0x7f0c98006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:41.199 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.198+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0c8c012b20 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.404 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.403+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0c9c108140 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.405 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.404+0000 7f0c91ffb700 1 -- 172.21.15.175:0/1755208797 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0c8c012b20 con 0x7f0c9c0f4b00 2023-12-06T03:26:41.405 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:41.405 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0c8802ff50 msgr2=0x7f0c88032410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0c8802ff50 0x7f0c88032410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0c980096a0 tx=0x7f0c98006b40).stop 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 msgr2=0x7f0c9c106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c106710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f0c8c013510 tx=0x7f0c8c00b040).stop 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 shutdown_connections 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0c8802ff50 0x7f0c88032410 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 --2- 172.21.15.175:0/1755208797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c9c0f4b00 0x7f0c9c106710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 >> 172.21.15.175:0/1755208797 conn(0x7f0c9c0f0260 msgr2=0x7f0c9c0f0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 shutdown_connections 2023-12-06T03:26:41.407 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:41.405+0000 7f0ca3506700 1 -- 172.21.15.175:0/1755208797 wait complete. 2023-12-06T03:26:41.408 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:40.453675+0000 mgr.smithi055.qjkkyb (mgr.14164) 164 : cluster [DBG] pgmap v153: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:41.404825+0000 mon.smithi055 (mon.0) 239 : audit [DBG] from='client.? 172.21.15.175:0/1755208797' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:42.800 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:42.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 -- 172.21.15.175:0/1066329133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 msgr2=0x7fc1000f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 --2- 172.21.15.175:0/1066329133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc1000f5830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fc0f4009fb0 tx=0x7fc0f4009b30).stop 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 -- 172.21.15.175:0/1066329133 shutdown_connections 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 --2- 172.21.15.175:0/1066329133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc1000f5830 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 -- 172.21.15.175:0/1066329133 >> 172.21.15.175:0/1066329133 conn(0x7fc1000f0b70 msgr2=0x7fc1000f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 -- 172.21.15.175:0/1066329133 shutdown_connections 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.923+0000 7fc105f72700 1 -- 172.21.15.175:0/1066329133 wait complete. 2023-12-06T03:26:43.925 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc105f72700 1 Processor -- start 2023-12-06T03:26:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc105f72700 1 -- start start 2023-12-06T03:26:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc105f72700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc100105630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc105f72700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100105b20 con 0x7fc1000f5410 2023-12-06T03:26:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc100105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:43.926 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc0ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc100105630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:44142/0 (socket says 172.21.15.175:44142) 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.924+0000 7fc0ff7fe700 1 -- 172.21.15.175:0/608910940 learned_addr learned my addr 172.21.15.175:0/608910940 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc0ff7fe700 1 -- 172.21.15.175:0/608910940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4005040 con 0x7fc1000f5410 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc0ff7fe700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc100105630 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc0f4013510 tx=0x7fc0f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0f4009320 con 0x7fc1000f5410 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc100105d20 con 0x7fc1000f5410 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc0f4013f00 con 0x7fc1000f5410 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0f4014d90 con 0x7fc1000f5410 2023-12-06T03:26:43.927 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.925+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001061b0 con 0x7fc1000f5410 2023-12-06T03:26:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.926+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fc0f40095f0 con 0x7fc1000f5410 2023-12-06T03:26:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.926+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc100045ad0 con 0x7fc1000f5410 2023-12-06T03:26:43.929 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.926+0000 7fc0e7fff700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc0e802ff60 0x7fc0e8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.926+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fc0f4040fe0 con 0x7fc1000f5410 2023-12-06T03:26:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.926+0000 7fc0feffd700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc0e802ff60 0x7fc0e8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:43.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.926+0000 7fc0feffd700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc0e802ff60 0x7fc0e8032420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc0f00096a0 tx=0x7fc0f0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:43.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:43.931+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc0f4010230 con 0x7fc1000f5410 2023-12-06T03:26:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:42.453923+0000 mgr.smithi055.qjkkyb (mgr.14164) 165 : cluster [DBG] pgmap v154: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:44.137 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.136+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc1000fae60 con 0x7fc1000f5410 2023-12-06T03:26:44.138 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.137+0000 7fc0e7fff700 1 -- 172.21.15.175:0/608910940 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc0f4010230 con 0x7fc1000f5410 2023-12-06T03:26:44.138 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:44.138 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:44.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc0e802ff60 msgr2=0x7fc0e8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc0e802ff60 0x7fc0e8032420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fc0f00096a0 tx=0x7fc0f0006b40).stop 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 msgr2=0x7fc100105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc100105630 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fc0f4013510 tx=0x7fc0f400b040).stop 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 shutdown_connections 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc0e802ff60 0x7fc0e8032420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 --2- 172.21.15.175:0/608910940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1000f5410 0x7fc100105630 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 >> 172.21.15.175:0/608910940 conn(0x7fc1000f0b70 msgr2=0x7fc1000f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 shutdown_connections 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:44.138+0000 7fc105f72700 1 -- 172.21.15.175:0/608910940 wait complete. 2023-12-06T03:26:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:45.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:44.137766+0000 mon.smithi055 (mon.0) 240 : audit [DBG] from='client.? 172.21.15.175:0/608910940' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:45.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:44.454208+0000 mgr.smithi055.qjkkyb (mgr.14164) 166 : cluster [DBG] pgmap v155: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:45.551 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:45.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 -- 172.21.15.175:0/1312737518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 msgr2=0x7f89180f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 --2- 172.21.15.175:0/1312737518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f89180f5830 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f8908009fb0 tx=0x7f8908009b30).stop 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 -- 172.21.15.175:0/1312737518 shutdown_connections 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 --2- 172.21.15.175:0/1312737518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f89180f5830 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 -- 172.21.15.175:0/1312737518 >> 172.21.15.175:0/1312737518 conn(0x7f89180f0b70 msgr2=0x7f89180f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 -- 172.21.15.175:0/1312737518 shutdown_connections 2023-12-06T03:26:46.746 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.743+0000 7f89206ea700 1 -- 172.21.15.175:0/1312737518 wait complete. 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f89206ea700 1 Processor -- start 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f89206ea700 1 -- start start 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f89206ea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f891810b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f89206ea700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8918107240 con 0x7f89180f5410 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f891e486700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f891810b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f891e486700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f891810b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45102/0 (socket says 172.21.15.175:45102) 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.744+0000 7f891e486700 1 -- 172.21.15.175:0/1577729591 learned_addr learned my addr 172.21.15.175:0/1577729591 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f891e486700 1 -- 172.21.15.175:0/1577729591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8908005040 con 0x7f89180f5410 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f891e486700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f891810b850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f8908014000 tx=0x7f890800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89080095f0 con 0x7f89180f5410 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8918060220 con 0x7f89180f5410 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8908007e70 con 0x7f89180f5410 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8908014ea0 con 0x7f89180f5410 2023-12-06T03:26:46.747 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.745+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891810bd40 con 0x7f89180f5410 2023-12-06T03:26:46.748 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.746+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8908025070 con 0x7f89180f5410 2023-12-06T03:26:46.748 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.746+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8918045ad0 con 0x7f89180f5410 2023-12-06T03:26:46.748 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.746+0000 7f890effd700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f890402ff60 0x7f8904032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:46.748 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.746+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f89080409f0 con 0x7f89180f5410 2023-12-06T03:26:46.749 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.746+0000 7f891dc85700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f890402ff60 0x7f8904032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:46.749 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.747+0000 7f891dc85700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f890402ff60 0x7f8904032420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f89140096a0 tx=0x7f8914006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:46.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.751+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f890800f4e0 con 0x7f89180f5410 2023-12-06T03:26:46.958 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.957+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f89180fae60 con 0x7f89180f5410 2023-12-06T03:26:46.959 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.958+0000 7f890effd700 1 -- 172.21.15.175:0/1577729591 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f890800f300 con 0x7f89180f5410 2023-12-06T03:26:46.960 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:46.960 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:46.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f890402ff60 msgr2=0x7f8904032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:46.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f890402ff60 0x7f8904032420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f89140096a0 tx=0x7f8914006b40).stop 2023-12-06T03:26:46.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 msgr2=0x7f891810b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f891810b850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f8908014000 tx=0x7f890800b040).stop 2023-12-06T03:26:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 shutdown_connections 2023-12-06T03:26:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f890402ff60 0x7f8904032420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 --2- 172.21.15.175:0/1577729591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89180f5410 0x7f891810b850 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 >> 172.21.15.175:0/1577729591 conn(0x7f89180f0b70 msgr2=0x7f89180f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 shutdown_connections 2023-12-06T03:26:46.963 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:46.959+0000 7f89206ea700 1 -- 172.21.15.175:0/1577729591 wait complete. 2023-12-06T03:26:46.963 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:46.454465+0000 mgr.smithi055.qjkkyb (mgr.14164) 167 : cluster [DBG] pgmap v156: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:46.958656+0000 mon.smithi055 (mon.0) 241 : audit [DBG] from='client.? 172.21.15.175:0/1577729591' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:48.337 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:48.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:49.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.476+0000 7f36f9914700 1 -- 172.21.15.175:0/2235109947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f40f47c0 msgr2=0x7f36f40f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:49.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.476+0000 7f36f9914700 1 --2- 172.21.15.175:0/2235109947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f40f47c0 0x7f36f40f4be0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f36e8009fb0 tx=0x7f36e8009b30).stop 2023-12-06T03:26:49.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.476+0000 7f36f9914700 1 -- 172.21.15.175:0/2235109947 shutdown_connections 2023-12-06T03:26:49.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.476+0000 7f36f9914700 1 --2- 172.21.15.175:0/2235109947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f40f47c0 0x7f36f40f4be0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.476+0000 7f36f9914700 1 -- 172.21.15.175:0/2235109947 >> 172.21.15.175:0/2235109947 conn(0x7f36f40f02e0 msgr2=0x7f36f40f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.476+0000 7f36f9914700 1 -- 172.21.15.175:0/2235109947 shutdown_connections 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.477+0000 7f36f9914700 1 -- 172.21.15.175:0/2235109947 wait complete. 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.477+0000 7f36f9914700 1 Processor -- start 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.477+0000 7f36f9914700 1 -- start start 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.477+0000 7f36f9914700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 0x7f36f4108de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.477+0000 7f36f9914700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f41092d0 con 0x7f36f41089c0 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 0x7f36f4108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 0x7f36f4108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45116/0 (socket says 172.21.15.175:45116) 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f2ffd700 1 -- 172.21.15.175:0/2336903342 learned_addr learned my addr 172.21.15.175:0/2336903342 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f2ffd700 1 -- 172.21.15.175:0/2336903342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8005040 con 0x7f36f41089c0 2023-12-06T03:26:49.480 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f2ffd700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 0x7f36f4108de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f36e8009f80 tx=0x7f36e8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36e80136c0 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f41094d0 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36e80095f0 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36e8014420 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.478+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f4109fb0 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.479+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f4059100 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.479+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f36e8025060 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.479+0000 7f36db7fe700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f36dc02ff60 0x7f36dc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.479+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f36e8040e00 con 0x7f36f41089c0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.480+0000 7f36f27fc700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f36dc02ff60 0x7f36dc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:49.483 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.480+0000 7f36f27fc700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f36dc02ff60 0x7f36dc032420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f36e40096a0 tx=0x7f36e4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:49.486 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.485+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f36e800f280 con 0x7f36f41089c0 2023-12-06T03:26:49.638 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:48.454824+0000 mgr.smithi055.qjkkyb (mgr.14164) 168 : cluster [DBG] pgmap v157: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:49.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.689+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f36f410a770 con 0x7f36f41089c0 2023-12-06T03:26:49.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.690+0000 7f36db7fe700 1 -- 172.21.15.175:0/2336903342 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f36e800f280 con 0x7f36f41089c0 2023-12-06T03:26:49.691 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:49.691 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:49.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f36dc02ff60 msgr2=0x7f36dc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:49.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f36dc02ff60 0x7f36dc032420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f36e40096a0 tx=0x7f36e4006b40).stop 2023-12-06T03:26:49.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 msgr2=0x7f36f4108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:49.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 0x7f36f4108de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f36e8009f80 tx=0x7f36e8009b30).stop 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 shutdown_connections 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f36dc02ff60 0x7f36dc032420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 --2- 172.21.15.175:0/2336903342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36f41089c0 0x7f36f4108de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 >> 172.21.15.175:0/2336903342 conn(0x7f36f40f02e0 msgr2=0x7f36f40f0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 shutdown_connections 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:49.692+0000 7f36f9914700 1 -- 172.21.15.175:0/2336903342 wait complete. 2023-12-06T03:26:49.694 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:50.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:50 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:49.690857+0000 mon.smithi055 (mon.0) 242 : audit [DBG] from='client.? 172.21.15.175:0/2336903342' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:50.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:50 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:50.485569+0000 mon.smithi055 (mon.0) 243 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' 2023-12-06T03:26:51.085 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:51.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:50.455108+0000 mgr.smithi055.qjkkyb (mgr.14164) 169 : cluster [DBG] pgmap v158: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:26:50.487281+0000 mgr.smithi055.qjkkyb (mgr.14164) 170 : cephadm [INF] Deploying daemon node-exporter.smithi055 on smithi055 2023-12-06T03:26:52.248 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 -- 172.21.15.175:0/2010765379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 msgr2=0x7f57a00f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:52.248 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 --2- 172.21.15.175:0/2010765379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a00f55c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f5790009fb0 tx=0x7f5790009b30).stop 2023-12-06T03:26:52.248 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 -- 172.21.15.175:0/2010765379 shutdown_connections 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 --2- 172.21.15.175:0/2010765379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a00f55c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 -- 172.21.15.175:0/2010765379 >> 172.21.15.175:0/2010765379 conn(0x7f57a00f09a0 msgr2=0x7f57a00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 -- 172.21.15.175:0/2010765379 shutdown_connections 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 -- 172.21.15.175:0/2010765379 wait complete. 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.246+0000 7f57a51bc700 1 Processor -- start 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.247+0000 7f57a51bc700 1 -- start start 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.247+0000 7f57a51bc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a01091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.247+0000 7f57a51bc700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a0060220 con 0x7f57a00f51a0 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.247+0000 7f579ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a01091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.247+0000 7f579ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a01091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45130/0 (socket says 172.21.15.175:45130) 2023-12-06T03:26:52.249 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.247+0000 7f579ffff700 1 -- 172.21.15.175:0/3273368881 learned_addr learned my addr 172.21.15.175:0/3273368881 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f579ffff700 1 -- 172.21.15.175:0/3273368881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5790005040 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f579ffff700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a01091d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f5790013510 tx=0x7f579000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5790009320 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a0060420 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a0109880 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5790013f00 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.248+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5790014d90 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.249+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f5790014420 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.249+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a0059100 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.249+0000 7f579cff9700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578802ff60 0x7f5788032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.249+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5790041310 con 0x7f57a00f51a0 2023-12-06T03:26:52.251 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.249+0000 7f579f7fe700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578802ff60 0x7f5788032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:52.252 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.250+0000 7f579f7fe700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578802ff60 0x7f5788032420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f57940096a0 tx=0x7f5794006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:52.255 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.254+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5790012b20 con 0x7f57a00f51a0 2023-12-06T03:26:52.462 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.461+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f57a0045ad0 con 0x7f57a00f51a0 2023-12-06T03:26:52.463 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.461+0000 7f579cff9700 1 -- 172.21.15.175:0/3273368881 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5790012b20 con 0x7f57a00f51a0 2023-12-06T03:26:52.463 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:52.463 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578802ff60 msgr2=0x7f5788032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578802ff60 0x7f5788032420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f57940096a0 tx=0x7f5794006b40).stop 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 msgr2=0x7f57a01091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a01091d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f5790013510 tx=0x7f579000b040).stop 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 shutdown_connections 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578802ff60 0x7f5788032420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 --2- 172.21.15.175:0/3273368881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57a00f51a0 0x7f57a01091d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 >> 172.21.15.175:0/3273368881 conn(0x7f57a00f09a0 msgr2=0x7f57a00f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.463+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 shutdown_connections 2023-12-06T03:26:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:52.464+0000 7f57a51bc700 1 -- 172.21.15.175:0/3273368881 wait complete. 2023-12-06T03:26:52.466 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:52.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:52.462689+0000 mon.smithi055 (mon.0) 244 : audit [DBG] from='client.? 172.21.15.175:0/3273368881' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:53.865 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:53.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:52.455361+0000 mgr.smithi055.qjkkyb (mgr.14164) 171 : cluster [DBG] pgmap v159: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:55.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.069+0000 7f4ee99ae700 1 -- 172.21.15.175:0/571826479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 msgr2=0x7f4ee4069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:55.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.069+0000 7f4ee99ae700 1 --2- 172.21.15.175:0/571826479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee4069ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f4ed8009fb0 tx=0x7f4ed8009b30).stop 2023-12-06T03:26:55.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 -- 172.21.15.175:0/571826479 shutdown_connections 2023-12-06T03:26:55.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 --2- 172.21.15.175:0/571826479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee4069ec0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:55.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 -- 172.21.15.175:0/571826479 >> 172.21.15.175:0/571826479 conn(0x7f4ee40f1aa0 msgr2=0x7f4ee40f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 -- 172.21.15.175:0/571826479 shutdown_connections 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 -- 172.21.15.175:0/571826479 wait complete. 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 Processor -- start 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 -- start start 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee410cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.070+0000 7f4ee99ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee410cfd0 con 0x7f4ee406b920 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee89ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee410cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee89ac700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee410cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:45142/0 (socket says 172.21.15.175:45142) 2023-12-06T03:26:55.074 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee89ac700 1 -- 172.21.15.175:0/700019315 learned_addr learned my addr 172.21.15.175:0/700019315 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee89ac700 1 -- 172.21.15.175:0/700019315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8005040 con 0x7f4ee406b920 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee89ac700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee410cae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f4ed8013510 tx=0x7f4ed800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ed8009320 con 0x7f4ee406b920 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee410d1d0 con 0x7f4ee406b920 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ed8013f00 con 0x7f4ee406b920 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4ed8014d90 con 0x7f4ee406b920 2023-12-06T03:26:55.075 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.071+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee410d660 con 0x7f4ee406b920 2023-12-06T03:26:55.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.072+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f4ed8014420 con 0x7f4ee406b920 2023-12-06T03:26:55.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.072+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee4045ad0 con 0x7f4ee406b920 2023-12-06T03:26:55.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.072+0000 7f4ee17fa700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4ed402ff60 0x7f4ed4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:55.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.072+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4ed80410c0 con 0x7f4ee406b920 2023-12-06T03:26:55.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.073+0000 7f4ee3fff700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4ed402ff60 0x7f4ed4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:55.077 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.073+0000 7f4ee3fff700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4ed402ff60 0x7f4ed4032420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f4ed00096a0 tx=0x7f4ed0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:55.079 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.077+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4ed80095f0 con 0x7f4ee406b920 2023-12-06T03:26:55.284 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.283+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ee410e510 con 0x7f4ee406b920 2023-12-06T03:26:55.285 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.283+0000 7f4ee17fa700 1 -- 172.21.15.175:0/700019315 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4ed801db70 con 0x7f4ee406b920 2023-12-06T03:26:55.285 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:55.285 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:55.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4ed402ff60 msgr2=0x7f4ed4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:55.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4ed402ff60 0x7f4ed4032420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f4ed00096a0 tx=0x7f4ed0006b40).stop 2023-12-06T03:26:55.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 msgr2=0x7f4ee410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:55.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee410cae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f4ed8013510 tx=0x7f4ed800b040).stop 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 shutdown_connections 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4ed402ff60 0x7f4ed4032420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 --2- 172.21.15.175:0/700019315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee406b920 0x7f4ee410cae0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 >> 172.21.15.175:0/700019315 conn(0x7f4ee40f1aa0 msgr2=0x7f4ee40f86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 shutdown_connections 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:55.285+0000 7f4ee99ae700 1 -- 172.21.15.175:0/700019315 wait complete. 2023-12-06T03:26:55.287 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:54.455691+0000 mgr.smithi055.qjkkyb (mgr.14164) 172 : cluster [DBG] pgmap v160: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:55.284540+0000 mon.smithi055 (mon.0) 245 : audit [DBG] from='client.? 172.21.15.175:0/700019315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:56.638 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:56.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:57.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:56.456032+0000 mgr.smithi055.qjkkyb (mgr.14164) 173 : cluster [DBG] pgmap v161: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:26:57.780 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 -- 172.21.15.175:0/3704409380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 msgr2=0x7f5adc0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:57.780 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 --2- 172.21.15.175:0/3704409380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc0f55c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f5acc009fb0 tx=0x7f5acc009b30).stop 2023-12-06T03:26:57.780 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 -- 172.21.15.175:0/3704409380 shutdown_connections 2023-12-06T03:26:57.780 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 --2- 172.21.15.175:0/3704409380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc0f55c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:57.781 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 -- 172.21.15.175:0/3704409380 >> 172.21.15.175:0/3704409380 conn(0x7f5adc0f09a0 msgr2=0x7f5adc0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:57.781 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 -- 172.21.15.175:0/3704409380 shutdown_connections 2023-12-06T03:26:57.781 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 -- 172.21.15.175:0/3704409380 wait complete. 2023-12-06T03:26:57.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.779+0000 7f5ae1229700 1 Processor -- start 2023-12-06T03:26:57.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5ae1229700 1 -- start start 2023-12-06T03:26:57.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5ae1229700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc1091b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:57.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5ae1229700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc060240 con 0x7f5adc0f51a0 2023-12-06T03:26:57.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5adbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc1091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:57.782 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5adbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc1091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53156/0 (socket says 172.21.15.175:53156) 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5adbfff700 1 -- 172.21.15.175:0/203542781 learned_addr learned my addr 172.21.15.175:0/203542781 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.780+0000 7f5adbfff700 1 -- 172.21.15.175:0/203542781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc005040 con 0x7f5adc0f51a0 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.781+0000 7f5adbfff700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc1091b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f5acc013510 tx=0x7f5acc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.781+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5acc009320 con 0x7f5adc0f51a0 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.781+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc060440 con 0x7f5adc0f51a0 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.781+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5acc013f00 con 0x7f5adc0f51a0 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.781+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5acc014dc0 con 0x7f5adc0f51a0 2023-12-06T03:26:57.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.781+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc109860 con 0x7f5adc0f51a0 2023-12-06T03:26:57.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.782+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f5acc0095f0 con 0x7f5adc0f51a0 2023-12-06T03:26:57.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.782+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5adc045ad0 con 0x7f5adc0f51a0 2023-12-06T03:26:57.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.782+0000 7f5ad8ff9700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ac402ff60 0x7f5ac4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:26:57.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.782+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5acc041000 con 0x7f5adc0f51a0 2023-12-06T03:26:57.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.782+0000 7f5adb7fe700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ac402ff60 0x7f5ac4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:26:57.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.782+0000 7f5adb7fe700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ac402ff60 0x7f5ac4032420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5ad00096a0 tx=0x7f5ad0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:26:57.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.787+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5acc01c790 con 0x7f5adc0f51a0 2023-12-06T03:26:57.995 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.993+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5adc10a080 con 0x7f5adc0f51a0 2023-12-06T03:26:57.995 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.994+0000 7f5ad8ff9700 1 -- 172.21.15.175:0/203542781 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5acc010230 con 0x7f5adc0f51a0 2023-12-06T03:26:57.995 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:26:57.995 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ac402ff60 msgr2=0x7f5ac4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ac402ff60 0x7f5ac4032420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f5ad00096a0 tx=0x7f5ad0006b40).stop 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 msgr2=0x7f5adc1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc1091b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f5acc013510 tx=0x7f5acc00b040).stop 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 shutdown_connections 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ac402ff60 0x7f5ac4032420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 --2- 172.21.15.175:0/203542781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5adc0f51a0 0x7f5adc1091b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.995+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 >> 172.21.15.175:0/203542781 conn(0x7f5adc0f09a0 msgr2=0x7f5adc0f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.996+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 shutdown_connections 2023-12-06T03:26:57.997 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:26:57.996+0000 7f5ae1229700 1 -- 172.21.15.175:0/203542781 wait complete. 2023-12-06T03:26:57.998 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:26:58.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:58 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:26:57.994974+0000 mon.smithi055 (mon.0) 246 : audit [DBG] from='client.? 172.21.15.175:0/203542781' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:26:59.387 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:26:59.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:26:59.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:26:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:26:58.456297+0000 mgr.smithi055.qjkkyb (mgr.14164) 174 : cluster [DBG] pgmap v162: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:00.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- 172.21.15.175:0/2885864835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 msgr2=0x7fca0c0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:00.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 --2- 172.21.15.175:0/2885864835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c0f55c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fca08009fb0 tx=0x7fca08009b30).stop 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- 172.21.15.175:0/2885864835 shutdown_connections 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 --2- 172.21.15.175:0/2885864835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c0f55c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- 172.21.15.175:0/2885864835 >> 172.21.15.175:0/2885864835 conn(0x7fca0c0f09a0 msgr2=0x7fca0c0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- 172.21.15.175:0/2885864835 shutdown_connections 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- 172.21.15.175:0/2885864835 wait complete. 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 Processor -- start 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- start start 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c1064e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.570+0000 7fca13271700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1069d0 con 0x7fca0c0f51a0 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca1226f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c1064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca1226f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c1064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53174/0 (socket says 172.21.15.175:53174) 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca1226f700 1 -- 172.21.15.175:0/1944413695 learned_addr learned my addr 172.21.15.175:0/1944413695 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca1226f700 1 -- 172.21.15.175:0/1944413695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08005040 con 0x7fca0c0f51a0 2023-12-06T03:27:00.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca1226f700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c1064e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fca08013510 tx=0x7fca0800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:00.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca08009320 con 0x7fca0c0f51a0 2023-12-06T03:27:00.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c106bd0 con 0x7fca0c0f51a0 2023-12-06T03:27:00.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca08013f00 con 0x7fca0c0f51a0 2023-12-06T03:27:00.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca08014dc0 con 0x7fca0c0f51a0 2023-12-06T03:27:00.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.571+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c107060 con 0x7fca0c0f51a0 2023-12-06T03:27:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.572+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fca080095f0 con 0x7fca0c0f51a0 2023-12-06T03:27:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.572+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c045ad0 con 0x7fca0c0f51a0 2023-12-06T03:27:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.572+0000 7fca02ffd700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc9f802ff60 0x7fc9f8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.572+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fca08041100 con 0x7fca0c0f51a0 2023-12-06T03:27:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.572+0000 7fca11a6e700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc9f802ff60 0x7fc9f8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:00.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.573+0000 7fca11a6e700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc9f802ff60 0x7fc9f8032420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fc9fc0096a0 tx=0x7fc9fc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:00.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.577+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fca08014420 con 0x7fca0c0f51a0 2023-12-06T03:27:00.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.782+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fca0c107f10 con 0x7fca0c0f51a0 2023-12-06T03:27:00.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.783+0000 7fca02ffd700 1 -- 172.21.15.175:0/1944413695 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fca08010230 con 0x7fca0c0f51a0 2023-12-06T03:27:00.784 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:00.784 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.784+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc9f802ff60 msgr2=0x7fc9f8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.784+0000 7fca13271700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc9f802ff60 0x7fc9f8032420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fc9fc0096a0 tx=0x7fc9fc006b40).stop 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 msgr2=0x7fca0c1064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c1064e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fca08013510 tx=0x7fca0800b040).stop 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 shutdown_connections 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fc9f802ff60 0x7fc9f8032420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 --2- 172.21.15.175:0/1944413695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca0c0f51a0 0x7fca0c1064e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 >> 172.21.15.175:0/1944413695 conn(0x7fca0c0f09a0 msgr2=0x7fca0c05fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 shutdown_connections 2023-12-06T03:27:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:00.785+0000 7fca13271700 1 -- 172.21.15.175:0/1944413695 wait complete. 2023-12-06T03:27:00.787 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:00.456549+0000 mgr.smithi055.qjkkyb (mgr.14164) 175 : cluster [DBG] pgmap v163: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:01.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:00.784011+0000 mon.smithi055 (mon.0) 247 : audit [DBG] from='client.? 172.21.15.175:0/1944413695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:02.168 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:02.168 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:03.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.283+0000 7feafc09f700 1 -- 172.21.15.175:0/952451818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 msgr2=0x7feaf40f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:03.286 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.283+0000 7feafc09f700 1 --2- 172.21.15.175:0/952451818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf40f6930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7feaf0009fb0 tx=0x7feaf0009b30).stop 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 -- 172.21.15.175:0/952451818 shutdown_connections 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 --2- 172.21.15.175:0/952451818 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf40f6930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 -- 172.21.15.175:0/952451818 >> 172.21.15.175:0/952451818 conn(0x7feaf40f1c70 msgr2=0x7feaf40f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 -- 172.21.15.175:0/952451818 shutdown_connections 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 -- 172.21.15.175:0/952451818 wait complete. 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 Processor -- start 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 -- start start 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feafc09f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4069e20 con 0x7feaf40f6510 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feaf9e3b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feaf9e3b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53204/0 (socket says 172.21.15.175:53204) 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.284+0000 7feaf9e3b700 1 -- 172.21.15.175:0/4119054253 learned_addr learned my addr 172.21.15.175:0/4119054253 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feaf9e3b700 1 -- 172.21.15.175:0/4119054253 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0005040 con 0x7feaf40f6510 2023-12-06T03:27:03.287 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feaf9e3b700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf406b9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7feaf0013510 tx=0x7feaf000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:03.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feaf0009320 con 0x7feaf40f6510 2023-12-06T03:27:03.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf4069fc0 con 0x7feaf40f6510 2023-12-06T03:27:03.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feaf0013f00 con 0x7feaf40f6510 2023-12-06T03:27:03.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feaf0014d90 con 0x7feaf40f6510 2023-12-06T03:27:03.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.285+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf406a450 con 0x7feaf40f6510 2023-12-06T03:27:03.288 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.286+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7feaf0014420 con 0x7feaf40f6510 2023-12-06T03:27:03.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.286+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf4059100 con 0x7feaf40f6510 2023-12-06T03:27:03.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.286+0000 7feaea7fc700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feae002ff60 0x7feae0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:03.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.286+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7feaf0040ff0 con 0x7feaf40f6510 2023-12-06T03:27:03.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.286+0000 7feaf963a700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feae002ff60 0x7feae0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:03.289 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.287+0000 7feaf963a700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feae002ff60 0x7feae0032420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7feae40096a0 tx=0x7feae4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:03.292 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.291+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7feaf00095f0 con 0x7feaf40f6510 2023-12-06T03:27:03.500 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.499+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feaf40fbf60 con 0x7feaf40f6510 2023-12-06T03:27:03.501 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.500+0000 7feaea7fc700 1 -- 172.21.15.175:0/4119054253 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7feaf001db70 con 0x7feaf40f6510 2023-12-06T03:27:03.502 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:03.502 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:03.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feae002ff60 msgr2=0x7feae0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:03.503 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feae002ff60 0x7feae0032420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7feae40096a0 tx=0x7feae4006b40).stop 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 msgr2=0x7feaf406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf406b9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7feaf0013510 tx=0x7feaf000b040).stop 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 shutdown_connections 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7feae002ff60 0x7feae0032420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 --2- 172.21.15.175:0/4119054253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feaf40f6510 0x7feaf406b9a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 >> 172.21.15.175:0/4119054253 conn(0x7feaf40f1c70 msgr2=0x7feaf40f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 shutdown_connections 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:03.502+0000 7feafc09f700 1 -- 172.21.15.175:0/4119054253 wait complete. 2023-12-06T03:27:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:02.456852+0000 mgr.smithi055.qjkkyb (mgr.14164) 176 : cluster [DBG] pgmap v164: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:03.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:03.501107+0000 mon.smithi055 (mon.0) 248 : audit [DBG] from='client.? 172.21.15.175:0/4119054253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:04.892 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:04.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:04.457161+0000 mgr.smithi055.qjkkyb (mgr.14164) 177 : cluster [DBG] pgmap v165: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.044+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2797779136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 msgr2=0x7fcad80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:06.046 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.044+0000 7fcadd5c4700 1 --2- 172.21.15.175:0/2797779136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad80f55c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fcac8009fb0 tx=0x7fcac8009b30).stop 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2797779136 shutdown_connections 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 --2- 172.21.15.175:0/2797779136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad80f55c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2797779136 >> 172.21.15.175:0/2797779136 conn(0x7fcad80f09a0 msgr2=0x7fcad80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2797779136 shutdown_connections 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2797779136 wait complete. 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 Processor -- start 2023-12-06T03:27:06.047 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.045+0000 7fcadd5c4700 1 -- start start 2023-12-06T03:27:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.046+0000 7fcadd5c4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad81053b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.046+0000 7fcadd5c4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81058a0 con 0x7fcad80f51a0 2023-12-06T03:27:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.046+0000 7fcad7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad81053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.046+0000 7fcad7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad81053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:50032/0 (socket says 172.21.15.175:50032) 2023-12-06T03:27:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.046+0000 7fcad7fff700 1 -- 172.21.15.175:0/2000957545 learned_addr learned my addr 172.21.15.175:0/2000957545 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:06.048 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.046+0000 7fcad7fff700 1 -- 172.21.15.175:0/2000957545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8005040 con 0x7fcad80f51a0 2023-12-06T03:27:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.047+0000 7fcad7fff700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad81053b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fcac8013510 tx=0x7fcac800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.047+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcac8009320 con 0x7fcad80f51a0 2023-12-06T03:27:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.047+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcac8013f00 con 0x7fcad80f51a0 2023-12-06T03:27:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.047+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad8105aa0 con 0x7fcad80f51a0 2023-12-06T03:27:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.047+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcac8014dc0 con 0x7fcad80f51a0 2023-12-06T03:27:06.049 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.047+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad8105f30 con 0x7fcad80f51a0 2023-12-06T03:27:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.048+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fcac801d3d0 con 0x7fcad80f51a0 2023-12-06T03:27:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.048+0000 7fcad4ff9700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcac002ff60 0x7fcac0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.048+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fcac80410c0 con 0x7fcad80f51a0 2023-12-06T03:27:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.048+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad8045ad0 con 0x7fcad80f51a0 2023-12-06T03:27:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.048+0000 7fcad77fe700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcac002ff60 0x7fcac0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:06.050 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.049+0000 7fcad77fe700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcac002ff60 0x7fcac0032420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcacc0096a0 tx=0x7fcacc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:06.054 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.053+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcac8012920 con 0x7fcad80f51a0 2023-12-06T03:27:06.262 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.261+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcad8106c50 con 0x7fcad80f51a0 2023-12-06T03:27:06.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.262+0000 7fcad4ff9700 1 -- 172.21.15.175:0/2000957545 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcac802b020 con 0x7fcad80f51a0 2023-12-06T03:27:06.263 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:06.263 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcac002ff60 msgr2=0x7fcac0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcac002ff60 0x7fcac0032420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fcacc0096a0 tx=0x7fcacc006b40).stop 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 msgr2=0x7fcad81053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad81053b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fcac8013510 tx=0x7fcac800b040).stop 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 shutdown_connections 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcac002ff60 0x7fcac0032420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 --2- 172.21.15.175:0/2000957545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcad80f51a0 0x7fcad81053b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 >> 172.21.15.175:0/2000957545 conn(0x7fcad80f09a0 msgr2=0x7fcad80f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 shutdown_connections 2023-12-06T03:27:06.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:06.264+0000 7fcadd5c4700 1 -- 172.21.15.175:0/2000957545 wait complete. 2023-12-06T03:27:06.266 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:06.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:06 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:06.263067+0000 mon.smithi055 (mon.0) 249 : audit [DBG] from='client.? 172.21.15.175:0/2000957545' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:07.656 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:07.657 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:07.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:06.457504+0000 mgr.smithi055.qjkkyb (mgr.14164) 178 : cluster [DBG] pgmap v166: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 -- 172.21.15.175:0/3902845786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 msgr2=0x7f42780fb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 --2- 172.21.15.175:0/3902845786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42780fb550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f4260009fb0 tx=0x7f4260009b30).stop 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 -- 172.21.15.175:0/3902845786 shutdown_connections 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 --2- 172.21.15.175:0/3902845786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42780fb550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 -- 172.21.15.175:0/3902845786 >> 172.21.15.175:0/3902845786 conn(0x7f42780f0b70 msgr2=0x7f42780f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 -- 172.21.15.175:0/3902845786 shutdown_connections 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.875+0000 7f427dd6b700 1 -- 172.21.15.175:0/3902845786 wait complete. 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f427dd6b700 1 Processor -- start 2023-12-06T03:27:08.877 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f427dd6b700 1 -- start start 2023-12-06T03:27:08.878 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f427dd6b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42781088c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:08.878 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f427dd6b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4278108db0 con 0x7f4278060220 2023-12-06T03:27:08.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f42777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42781088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:08.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f42777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42781088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:50048/0 (socket says 172.21.15.175:50048) 2023-12-06T03:27:08.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f42777fe700 1 -- 172.21.15.175:0/1257644621 learned_addr learned my addr 172.21.15.175:0/1257644621 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:08.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.876+0000 7f42777fe700 1 -- 172.21.15.175:0/1257644621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260005040 con 0x7f4278060220 2023-12-06T03:27:08.879 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.877+0000 7f42777fe700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42781088c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f4260013510 tx=0x7f426000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.877+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4260009320 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.877+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4278108fb0 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.877+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4260013f00 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.877+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4260014dc0 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.877+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4278109440 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.878+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f42600095f0 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.878+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4278045ad0 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.878+0000 7f426ffff700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f426402ff60 0x7f4264032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.878+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f42600410c0 con 0x7f4278060220 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.878+0000 7f4276ffd700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f426402ff60 0x7f4264032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:08.880 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.878+0000 7f4276ffd700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f426402ff60 0x7f4264032420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f42680096a0 tx=0x7f4268006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:08.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:08.883+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4260010230 con 0x7f4278060220 2023-12-06T03:27:09.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.087+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f427810a2f0 con 0x7f4278060220 2023-12-06T03:27:09.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.087+0000 7f426ffff700 1 -- 172.21.15.175:0/1257644621 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4260010230 con 0x7f4278060220 2023-12-06T03:27:09.089 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:09.089 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:09.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f426402ff60 msgr2=0x7f4264032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:09.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f426402ff60 0x7f4264032420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f42680096a0 tx=0x7f4268006b40).stop 2023-12-06T03:27:09.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 msgr2=0x7f42781088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42781088c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f4260013510 tx=0x7f426000b040).stop 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 shutdown_connections 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f426402ff60 0x7f4264032420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 --2- 172.21.15.175:0/1257644621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4278060220 0x7f42781088c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 >> 172.21.15.175:0/1257644621 conn(0x7f42780f0b70 msgr2=0x7f42780f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 shutdown_connections 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:09.089+0000 7f427dd6b700 1 -- 172.21.15.175:0/1257644621 wait complete. 2023-12-06T03:27:09.091 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:08.457928+0000 mgr.smithi055.qjkkyb (mgr.14164) 179 : cluster [DBG] pgmap v167: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:09.088601+0000 mon.smithi055 (mon.0) 250 : audit [DBG] from='client.? 172.21.15.175:0/1257644621' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:10.492 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:10.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:11.690 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 -- 172.21.15.175:0/1246218962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 msgr2=0x7fa3d00f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:11.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 --2- 172.21.15.175:0/1246218962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d00f6930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fa3c0009fb0 tx=0x7fa3c0009b30).stop 2023-12-06T03:27:11.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 -- 172.21.15.175:0/1246218962 shutdown_connections 2023-12-06T03:27:11.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 --2- 172.21.15.175:0/1246218962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d00f6930 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:11.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 -- 172.21.15.175:0/1246218962 >> 172.21.15.175:0/1246218962 conn(0x7fa3d00f1c70 msgr2=0x7fa3d00f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:11.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 -- 172.21.15.175:0/1246218962 shutdown_connections 2023-12-06T03:27:11.691 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.689+0000 7fa3d522f700 1 -- 172.21.15.175:0/1246218962 wait complete. 2023-12-06T03:27:11.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.690+0000 7fa3d522f700 1 Processor -- start 2023-12-06T03:27:11.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.690+0000 7fa3d522f700 1 -- start start 2023-12-06T03:27:11.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.690+0000 7fa3d522f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d0106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:11.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.690+0000 7fa3d522f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0106c20 con 0x7fa3d00f6510 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3ced9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d0106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3ced9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d0106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:50062/0 (socket says 172.21.15.175:50062) 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3ced9d700 1 -- 172.21.15.175:0/1040886801 learned_addr learned my addr 172.21.15.175:0/1040886801 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3ced9d700 1 -- 172.21.15.175:0/1040886801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0005040 con 0x7fa3d00f6510 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3ced9d700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d0106730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa3c0013510 tx=0x7fa3c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3c0009320 con 0x7fa3d00f6510 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d0106e20 con 0x7fa3d00f6510 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa3c0013f00 con 0x7fa3d00f6510 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3c0014d90 con 0x7fa3d00f6510 2023-12-06T03:27:11.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.691+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d01072b0 con 0x7fa3d00f6510 2023-12-06T03:27:11.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.692+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa3c00263a0 con 0x7fa3d00f6510 2023-12-06T03:27:11.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.692+0000 7fa3c77fe700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa3bc02ff60 0x7fa3bc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:11.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.692+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0045ad0 con 0x7fa3d00f6510 2023-12-06T03:27:11.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.692+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa3c0041020 con 0x7fa3d00f6510 2023-12-06T03:27:11.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.692+0000 7fa3ce59c700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa3bc02ff60 0x7fa3bc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:11.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.693+0000 7fa3ce59c700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa3bc02ff60 0x7fa3bc032420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa3b80096a0 tx=0x7fa3b8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:11.698 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.697+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa3c006d2b0 con 0x7fa3d00f6510 2023-12-06T03:27:11.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.904+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa3d00fc050 con 0x7fa3d00f6510 2023-12-06T03:27:11.906 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.905+0000 7fa3c77fe700 1 -- 172.21.15.175:0/1040886801 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa3c00126f0 con 0x7fa3d00f6510 2023-12-06T03:27:11.906 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:11.906 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa3bc02ff60 msgr2=0x7fa3bc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa3bc02ff60 0x7fa3bc032420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fa3b80096a0 tx=0x7fa3b8006b40).stop 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 msgr2=0x7fa3d0106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d0106730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fa3c0013510 tx=0x7fa3c000b040).stop 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 shutdown_connections 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa3bc02ff60 0x7fa3bc032420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 --2- 172.21.15.175:0/1040886801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3d00f6510 0x7fa3d0106730 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:11.908 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 >> 172.21.15.175:0/1040886801 conn(0x7fa3d00f1c70 msgr2=0x7fa3d00f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:11.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 shutdown_connections 2023-12-06T03:27:11.909 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:11.906+0000 7fa3d522f700 1 -- 172.21.15.175:0/1040886801 wait complete. 2023-12-06T03:27:11.909 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:10.458289+0000 mgr.smithi055.qjkkyb (mgr.14164) 180 : cluster [DBG] pgmap v168: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:12.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:11.905965+0000 mon.smithi055 (mon.0) 251 : audit [DBG] from='client.? 172.21.15.175:0/1040886801' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:13.275 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:13.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:12.458614+0000 mgr.smithi055.qjkkyb (mgr.14164) 181 : cluster [DBG] pgmap v169: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.415+0000 7f58d1b39700 1 -- 172.21.15.175:0/2559565981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 msgr2=0x7f58cc0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.415+0000 7f58d1b39700 1 --2- 172.21.15.175:0/2559565981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc0f5830 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f58c0009fb0 tx=0x7f58c0009b30).stop 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.415+0000 7f58d1b39700 1 -- 172.21.15.175:0/2559565981 shutdown_connections 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.416+0000 7f58d1b39700 1 --2- 172.21.15.175:0/2559565981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc0f5830 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.416+0000 7f58d1b39700 1 -- 172.21.15.175:0/2559565981 >> 172.21.15.175:0/2559565981 conn(0x7f58cc0f0b70 msgr2=0x7f58cc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.416+0000 7f58d1b39700 1 -- 172.21.15.175:0/2559565981 shutdown_connections 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.416+0000 7f58d1b39700 1 -- 172.21.15.175:0/2559565981 wait complete. 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.416+0000 7f58d1b39700 1 Processor -- start 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.416+0000 7f58d1b39700 1 -- start start 2023-12-06T03:27:14.418 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58d1b39700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:14.419 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58d1b39700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc060240 con 0x7f58cc0f5410 2023-12-06T03:27:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58cb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58cb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:50080/0 (socket says 172.21.15.175:50080) 2023-12-06T03:27:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58cb7fe700 1 -- 172.21.15.175:0/374686056 learned_addr learned my addr 172.21.15.175:0/374686056 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:14.420 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58cb7fe700 1 -- 172.21.15.175:0/374686056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c0005040 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58cb7fe700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc109400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f58c0014000 tx=0x7f58c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58c00095f0 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc060440 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58c0007e70 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.417+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58c0014ea0 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.418+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc109ab0 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.418+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f58c0014690 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.418+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc045ad0 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.418+0000 7f58b3fff700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f58b402ff60 0x7f58b4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.419+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f58c0040930 con 0x7f58cc0f5410 2023-12-06T03:27:14.421 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.419+0000 7f58caffd700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f58b402ff60 0x7f58b4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:14.422 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.419+0000 7f58caffd700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f58b402ff60 0x7f58b4032420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f58bc0096a0 tx=0x7f58bc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:14.424 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.423+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f58c000f300 con 0x7f58cc0f5410 2023-12-06T03:27:14.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.629+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58cc0fae60 con 0x7f58cc0f5410 2023-12-06T03:27:14.631 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.630+0000 7f58b3fff700 1 -- 172.21.15.175:0/374686056 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f58c000f300 con 0x7f58cc0f5410 2023-12-06T03:27:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:14.632 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f58b402ff60 msgr2=0x7f58b4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f58b402ff60 0x7f58b4032420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f58bc0096a0 tx=0x7f58bc006b40).stop 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 msgr2=0x7f58cc109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc109400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f58c0014000 tx=0x7f58c000b040).stop 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 shutdown_connections 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f58b402ff60 0x7f58b4032420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 --2- 172.21.15.175:0/374686056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58cc0f5410 0x7f58cc109400 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 >> 172.21.15.175:0/374686056 conn(0x7f58cc0f0b70 msgr2=0x7f58cc0f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 shutdown_connections 2023-12-06T03:27:14.633 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:14.632+0000 7f58d1b39700 1 -- 172.21.15.175:0/374686056 wait complete. 2023-12-06T03:27:14.634 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:15.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:14.458999+0000 mgr.smithi055.qjkkyb (mgr.14164) 182 : cluster [DBG] pgmap v170: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:15.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:14.631162+0000 mon.smithi055 (mon.0) 252 : audit [DBG] from='client.? 172.21.15.175:0/374686056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:16.046 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:16.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.215+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/3421319445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 msgr2=0x7fe7c80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.215+0000 7fe7cc9aa700 1 --2- 172.21.15.175:0/3421319445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c80f5830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fe7b0009fb0 tx=0x7fe7b0009b30).stop 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/3421319445 shutdown_connections 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 --2- 172.21.15.175:0/3421319445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c80f5830 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/3421319445 >> 172.21.15.175:0/3421319445 conn(0x7fe7c80f0b70 msgr2=0x7fe7c80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/3421319445 shutdown_connections 2023-12-06T03:27:17.218 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/3421319445 wait complete. 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 Processor -- start 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.216+0000 7fe7cc9aa700 1 -- start start 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7cc9aa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c81093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7cc9aa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c8060280 con 0x7fe7c80f5410 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c81093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7c659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c81093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42224/0 (socket says 172.21.15.175:42224) 2023-12-06T03:27:17.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7c659c700 1 -- 172.21.15.175:0/653997459 learned_addr learned my addr 172.21.15.175:0/653997459 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:17.220 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7c659c700 1 -- 172.21.15.175:0/653997459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0005040 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7c659c700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c81093f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fe7b0013510 tx=0x7fe7b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7b0009320 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c8060480 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7b0013f00 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7b0014d90 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.217+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c8109b20 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.218+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fe7b0014420 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.218+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c8045ad0 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.218+0000 7fe7beffd700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7b402ff60 0x7fe7b4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.219+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fe7b0040fb0 con 0x7fe7c80f5410 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.219+0000 7fe7c5d9b700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7b402ff60 0x7fe7b4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:17.221 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.219+0000 7fe7c5d9b700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7b402ff60 0x7fe7b4032420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe7b80096a0 tx=0x7fe7b8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:17.225 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.223+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe7b00095f0 con 0x7fe7c80f5410 2023-12-06T03:27:17.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.432+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe7c810a340 con 0x7fe7c80f5410 2023-12-06T03:27:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.433+0000 7fe7beffd700 1 -- 172.21.15.175:0/653997459 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe7b001db70 con 0x7fe7c80f5410 2023-12-06T03:27:17.434 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:17.434 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.434+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7b402ff60 msgr2=0x7fe7b4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7b402ff60 0x7fe7b4032420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fe7b80096a0 tx=0x7fe7b8006b40).stop 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 msgr2=0x7fe7c81093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c81093f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fe7b0013510 tx=0x7fe7b000b040).stop 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 shutdown_connections 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe7b402ff60 0x7fe7b4032420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 --2- 172.21.15.175:0/653997459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7c80f5410 0x7fe7c81093f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:17.436 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 >> 172.21.15.175:0/653997459 conn(0x7fe7c80f0b70 msgr2=0x7fe7c80f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:17.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 shutdown_connections 2023-12-06T03:27:17.437 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:17.435+0000 7fe7cc9aa700 1 -- 172.21.15.175:0/653997459 wait complete. 2023-12-06T03:27:17.437 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:16.459249+0000 mgr.smithi055.qjkkyb (mgr.14164) 183 : cluster [DBG] pgmap v171: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:17.434225+0000 mon.smithi055 (mon.0) 253 : audit [DBG] from='client.? 172.21.15.175:0/653997459' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:18.773 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:18.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:19.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/2543971147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 msgr2=0x7fb6c00f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:19.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 --2- 172.21.15.175:0/2543971147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c00f90f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fb6b4009fb0 tx=0x7fb6b4009b30).stop 2023-12-06T03:27:19.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/2543971147 shutdown_connections 2023-12-06T03:27:19.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 --2- 172.21.15.175:0/2543971147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c00f90f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:19.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/2543971147 >> 172.21.15.175:0/2543971147 conn(0x7fb6c00f0b70 msgr2=0x7fb6c00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:19.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/2543971147 shutdown_connections 2023-12-06T03:27:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.896+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/2543971147 wait complete. 2023-12-06T03:27:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c6c8d700 1 Processor -- start 2023-12-06T03:27:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c6c8d700 1 -- start start 2023-12-06T03:27:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c6c8d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c01066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:19.899 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c6c8d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c0106bd0 con 0x7fb6c0060280 2023-12-06T03:27:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c4a29700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c01066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c4a29700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c01066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42246/0 (socket says 172.21.15.175:42246) 2023-12-06T03:27:19.900 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.897+0000 7fb6c4a29700 1 -- 172.21.15.175:0/4247557234 learned_addr learned my addr 172.21.15.175:0/4247557234 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6c4a29700 1 -- 172.21.15.175:0/4247557234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4005040 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6c4a29700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c01066e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fb6b4013510 tx=0x7fb6b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6b4009320 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c0106dd0 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb6b4013f00 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6b4014dc0 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.898+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c0107260 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.899+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb6b40095f0 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.899+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c0045ad0 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.899+0000 7fb6bd7fa700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb6b002ff60 0x7fb6b0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.899+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb6b40410c0 con 0x7fb6c0060280 2023-12-06T03:27:19.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.899+0000 7fb6bffff700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb6b002ff60 0x7fb6b0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:19.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.900+0000 7fb6bffff700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb6b002ff60 0x7fb6b0032420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb6ac0096a0 tx=0x7fb6ac006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:19.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:19.904+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb6b4014420 con 0x7fb6c0060280 2023-12-06T03:27:19.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:18.459467+0000 mgr.smithi055.qjkkyb (mgr.14164) 184 : cluster [DBG] pgmap v172: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:20.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.112+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb6c0108110 con 0x7fb6c0060280 2023-12-06T03:27:20.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.112+0000 7fb6bd7fa700 1 -- 172.21.15.175:0/4247557234 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb6b4010230 con 0x7fb6c0060280 2023-12-06T03:27:20.114 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:20.114 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:20.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb6b002ff60 msgr2=0x7fb6b0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:20.115 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb6b002ff60 0x7fb6b0032420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb6ac0096a0 tx=0x7fb6ac006b40).stop 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 msgr2=0x7fb6c01066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c01066e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fb6b4013510 tx=0x7fb6b400b040).stop 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 shutdown_connections 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb6b002ff60 0x7fb6b0032420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 --2- 172.21.15.175:0/4247557234 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6c0060280 0x7fb6c01066e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 >> 172.21.15.175:0/4247557234 conn(0x7fb6c00f0b70 msgr2=0x7fb6c00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 shutdown_connections 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:20.114+0000 7fb6c6c8d700 1 -- 172.21.15.175:0/4247557234 wait complete. 2023-12-06T03:27:20.116 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:20.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:20 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:20.113425+0000 mon.smithi055 (mon.0) 254 : audit [DBG] from='client.? 172.21.15.175:0/4247557234' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:21.500 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:21.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:20.459802+0000 mgr.smithi055.qjkkyb (mgr.14164) 185 : cluster [DBG] pgmap v173: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:22.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- 172.21.15.175:0/2567304627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 msgr2=0x7f87500f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 --2- 172.21.15.175:0/2567304627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87500f55c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f8740009fb0 tx=0x7f8740009b30).stop 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- 172.21.15.175:0/2567304627 shutdown_connections 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 --2- 172.21.15.175:0/2567304627 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87500f55c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- 172.21.15.175:0/2567304627 >> 172.21.15.175:0/2567304627 conn(0x7f87500f09a0 msgr2=0x7f87500f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- 172.21.15.175:0/2567304627 shutdown_connections 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- 172.21.15.175:0/2567304627 wait complete. 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 Processor -- start 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- start start 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87501091b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.620+0000 7f87566d3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8750060240 con 0x7f87500f51a0 2023-12-06T03:27:22.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87556d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87501091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:22.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87556d1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87501091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42256/0 (socket says 172.21.15.175:42256) 2023-12-06T03:27:22.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87556d1700 1 -- 172.21.15.175:0/2842200346 learned_addr learned my addr 172.21.15.175:0/2842200346 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87556d1700 1 -- 172.21.15.175:0/2842200346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740005040 con 0x7f87500f51a0 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87556d1700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87501091b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f8740013510 tx=0x7f874000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8740009320 con 0x7f87500f51a0 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8750060440 con 0x7f87500f51a0 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8740013f00 con 0x7f87500f51a0 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8740014dc0 con 0x7f87500f51a0 2023-12-06T03:27:22.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.621+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8750109860 con 0x7f87500f51a0 2023-12-06T03:27:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.622+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f87400095f0 con 0x7f87500f51a0 2023-12-06T03:27:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.622+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8750045ad0 con 0x7f87500f51a0 2023-12-06T03:27:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.622+0000 7f8745ffb700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f873c02ff60 0x7f873c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.622+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8740041100 con 0x7f87500f51a0 2023-12-06T03:27:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.623+0000 7f8754ed0700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f873c02ff60 0x7f873c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.623+0000 7f8754ed0700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f873c02ff60 0x7f873c032420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f874c0096a0 tx=0x7f874c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:22.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.627+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8740014420 con 0x7f87500f51a0 2023-12-06T03:27:22.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.833+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f875010a080 con 0x7f87500f51a0 2023-12-06T03:27:22.835 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.834+0000 7f8745ffb700 1 -- 172.21.15.175:0/2842200346 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8740010230 con 0x7f87500f51a0 2023-12-06T03:27:22.835 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:22.836 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f873c02ff60 msgr2=0x7f873c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f873c02ff60 0x7f873c032420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f874c0096a0 tx=0x7f874c006b40).stop 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 msgr2=0x7f87501091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87501091b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f8740013510 tx=0x7f874000b040).stop 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 shutdown_connections 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f873c02ff60 0x7f873c032420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:22.837 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 --2- 172.21.15.175:0/2842200346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87500f51a0 0x7f87501091b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:22.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 >> 172.21.15.175:0/2842200346 conn(0x7f87500f09a0 msgr2=0x7f87500f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:22.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 shutdown_connections 2023-12-06T03:27:22.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:22.836+0000 7f87566d3700 1 -- 172.21.15.175:0/2842200346 wait complete. 2023-12-06T03:27:22.838 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:22.460076+0000 mgr.smithi055.qjkkyb (mgr.14164) 186 : cluster [DBG] pgmap v174: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:22.554744+0000 mon.smithi055 (mon.0) 255 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:27:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:22.557738+0000 mon.smithi055 (mon.0) 256 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:27:23.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:22.835215+0000 mon.smithi055 (mon.0) 257 : audit [DBG] from='client.? 172.21.15.175:0/2842200346' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:24.234 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:24.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 -- 172.21.15.175:0/1118364681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 msgr2=0x7f6cec0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 --2- 172.21.15.175:0/1118364681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec0f6930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f6ce0009fb0 tx=0x7f6ce0009b30).stop 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 -- 172.21.15.175:0/1118364681 shutdown_connections 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 --2- 172.21.15.175:0/1118364681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec0f6930 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 -- 172.21.15.175:0/1118364681 >> 172.21.15.175:0/1118364681 conn(0x7f6cec0f1c70 msgr2=0x7f6cec0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 -- 172.21.15.175:0/1118364681 shutdown_connections 2023-12-06T03:27:25.431 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.429+0000 7f6cf2053700 1 -- 172.21.15.175:0/1118364681 wait complete. 2023-12-06T03:27:25.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.430+0000 7f6cf2053700 1 Processor -- start 2023-12-06T03:27:25.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.430+0000 7f6cf2053700 1 -- start start 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.430+0000 7f6cf2053700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.430+0000 7f6cf2053700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec106c20 con 0x7f6cec0f6510 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6ceb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6ceb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51624/0 (socket says 172.21.15.175:51624) 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6ceb7fe700 1 -- 172.21.15.175:0/1304252540 learned_addr learned my addr 172.21.15.175:0/1304252540 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6ceb7fe700 1 -- 172.21.15.175:0/1304252540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0005040 con 0x7f6cec0f6510 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6ceb7fe700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec106730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f6ce0013510 tx=0x7f6ce000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce0009320 con 0x7f6cec0f6510 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec106e20 con 0x7f6cec0f6510 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ce0013f00 con 0x7f6cec0f6510 2023-12-06T03:27:25.433 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ce0014d90 con 0x7f6cec0f6510 2023-12-06T03:27:25.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.431+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec1072b0 con 0x7f6cec0f6510 2023-12-06T03:27:25.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.432+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6ce00095f0 con 0x7f6cec0f6510 2023-12-06T03:27:25.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.432+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec045ad0 con 0x7f6cec0f6510 2023-12-06T03:27:25.434 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.432+0000 7f6cd3fff700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6cd402ff60 0x7f6cd4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:25.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.432+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6ce0041000 con 0x7f6cec0f6510 2023-12-06T03:27:25.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.433+0000 7f6ceaffd700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6cd402ff60 0x7f6cd4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:25.435 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.433+0000 7f6ceaffd700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6cd402ff60 0x7f6cd4032420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6cdc0096a0 tx=0x7f6cdc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:25.439 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.437+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6ce0014420 con 0x7f6cec0f6510 2023-12-06T03:27:25.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:24.460395+0000 mgr.smithi055.qjkkyb (mgr.14164) 187 : cluster [DBG] pgmap v175: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:25.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.643+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6cec0fbf60 con 0x7f6cec0f6510 2023-12-06T03:27:25.644 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.643+0000 7f6cd3fff700 1 -- 172.21.15.175:0/1304252540 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6ce0010230 con 0x7f6cec0f6510 2023-12-06T03:27:25.645 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:25.645 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6cd402ff60 msgr2=0x7f6cd4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6cd402ff60 0x7f6cd4032420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f6cdc0096a0 tx=0x7f6cdc006b40).stop 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 msgr2=0x7f6cec106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec106730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f6ce0013510 tx=0x7f6ce000b040).stop 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 shutdown_connections 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6cd402ff60 0x7f6cd4032420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 --2- 172.21.15.175:0/1304252540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6cec0f6510 0x7f6cec106730 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 >> 172.21.15.175:0/1304252540 conn(0x7f6cec0f1c70 msgr2=0x7f6cec0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 shutdown_connections 2023-12-06T03:27:25.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:25.645+0000 7f6cf2053700 1 -- 172.21.15.175:0/1304252540 wait complete. 2023-12-06T03:27:25.649 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:26.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:25.644612+0000 mon.smithi055 (mon.0) 258 : audit [DBG] from='client.? 172.21.15.175:0/1304252540' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:27.029 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:27.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:27.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:26.460695+0000 mgr.smithi055.qjkkyb (mgr.14164) 188 : cluster [DBG] pgmap v176: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:28.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.160+0000 7f609d917700 1 -- 172.21.15.175:0/379765947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 msgr2=0x7f60980f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:28.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.160+0000 7f609d917700 1 --2- 172.21.15.175:0/379765947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f60980f6930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f6088009fb0 tx=0x7f6088009b30).stop 2023-12-06T03:27:28.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.160+0000 7f609d917700 1 -- 172.21.15.175:0/379765947 shutdown_connections 2023-12-06T03:27:28.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.160+0000 7f609d917700 1 --2- 172.21.15.175:0/379765947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f60980f6930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:28.162 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.161+0000 7f609d917700 1 -- 172.21.15.175:0/379765947 >> 172.21.15.175:0/379765947 conn(0x7f60980f1c70 msgr2=0x7f60980f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:28.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.161+0000 7f609d917700 1 -- 172.21.15.175:0/379765947 shutdown_connections 2023-12-06T03:27:28.163 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.161+0000 7f609d917700 1 -- 172.21.15.175:0/379765947 wait complete. 2023-12-06T03:27:28.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.161+0000 7f609d917700 1 Processor -- start 2023-12-06T03:27:28.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.161+0000 7f609d917700 1 -- start start 2023-12-06T03:27:28.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f609d917700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f6098106720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:28.164 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f609d917700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6098106c10 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f6096ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f6098106720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f6096ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f6098106720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51646/0 (socket says 172.21.15.175:51646) 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f6096ffd700 1 -- 172.21.15.175:0/2594170942 learned_addr learned my addr 172.21.15.175:0/2594170942 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f6096ffd700 1 -- 172.21.15.175:0/2594170942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088005040 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f6096ffd700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f6098106720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f6088013510 tx=0x7f608800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6088009320 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.162+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6098106e10 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.163+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6088013f00 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.163+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6088014d90 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.163+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981072a0 con 0x7f60980f6510 2023-12-06T03:27:28.165 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.163+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f60880095f0 con 0x7f60980f6510 2023-12-06T03:27:28.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.163+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098045ad0 con 0x7f60980f6510 2023-12-06T03:27:28.166 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.164+0000 7f608f7fe700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f608402ff60 0x7f6084032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:28.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.164+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6088040fc0 con 0x7f60980f6510 2023-12-06T03:27:28.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.164+0000 7f60967fc700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f608402ff60 0x7f6084032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:28.167 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.164+0000 7f60967fc700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f608402ff60 0x7f6084032420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f60800096a0 tx=0x7f6080006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:28.170 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.169+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6088014420 con 0x7f60980f6510 2023-12-06T03:27:28.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.376+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f60981080f0 con 0x7f60980f6510 2023-12-06T03:27:28.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.377+0000 7f608f7fe700 1 -- 172.21.15.175:0/2594170942 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6088010230 con 0x7f60980f6510 2023-12-06T03:27:28.378 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:28.378 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f608402ff60 msgr2=0x7f6084032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f608402ff60 0x7f6084032420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f60800096a0 tx=0x7f6080006b40).stop 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 msgr2=0x7f6098106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f6098106720 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f6088013510 tx=0x7f608800b040).stop 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 shutdown_connections 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f608402ff60 0x7f6084032420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:28.380 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 --2- 172.21.15.175:0/2594170942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60980f6510 0x7f6098106720 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:28.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 >> 172.21.15.175:0/2594170942 conn(0x7f60980f1c70 msgr2=0x7f60980f27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:28.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 shutdown_connections 2023-12-06T03:27:28.381 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:28.379+0000 7f609d917700 1 -- 172.21.15.175:0/2594170942 wait complete. 2023-12-06T03:27:28.381 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:28.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:28.377966+0000 mon.smithi055 (mon.0) 259 : audit [DBG] from='client.? 172.21.15.175:0/2594170942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:29.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:28.461045+0000 mgr.smithi055.qjkkyb (mgr.14164) 189 : cluster [DBG] pgmap v177: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:29.762 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:29.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- 172.21.15.175:0/2131834272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 msgr2=0x7f11c80f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 --2- 172.21.15.175:0/2131834272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c80f66c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f11b0009fb0 tx=0x7f11b0009b30).stop 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- 172.21.15.175:0/2131834272 shutdown_connections 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 --2- 172.21.15.175:0/2131834272 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c80f66c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- 172.21.15.175:0/2131834272 >> 172.21.15.175:0/2131834272 conn(0x7f11c80f1aa0 msgr2=0x7f11c80f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- 172.21.15.175:0/2131834272 shutdown_connections 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- 172.21.15.175:0/2131834272 wait complete. 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 Processor -- start 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- start start 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c81064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.898+0000 7f11c759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81069a0 con 0x7f11c80f62a0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c81064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c81064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51664/0 (socket says 172.21.15.175:51664) 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c659c700 1 -- 172.21.15.175:0/3980539877 learned_addr learned my addr 172.21.15.175:0/3980539877 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c659c700 1 -- 172.21.15.175:0/3980539877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b0005040 con 0x7f11c80f62a0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c659c700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c81064b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f11b0013510 tx=0x7f11b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11b0009320 con 0x7f11c80f62a0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c8106ba0 con 0x7f11c80f62a0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11b0013f00 con 0x7f11c80f62a0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f11b0014dc0 con 0x7f11c80f62a0 2023-12-06T03:27:30.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.899+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c8107030 con 0x7f11c80f62a0 2023-12-06T03:27:30.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.900+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8045ad0 con 0x7f11c80f62a0 2023-12-06T03:27:30.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.900+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f11b00095f0 con 0x7f11c80f62a0 2023-12-06T03:27:30.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.900+0000 7f11beffd700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f11b402ffb0 0x7f11b4032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:30.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.900+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f11b0040ff0 con 0x7f11c80f62a0 2023-12-06T03:27:30.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.900+0000 7f11c5d9b700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f11b402ffb0 0x7f11b4032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:30.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.901+0000 7f11c5d9b700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f11b402ffb0 0x7f11b4032470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f11b80096a0 tx=0x7f11b8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:30.906 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:30.905+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f11b006d2b0 con 0x7f11c80f62a0 2023-12-06T03:27:31.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.112+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f11c8107ee0 con 0x7f11c80f62a0 2023-12-06T03:27:31.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.113+0000 7f11beffd700 1 -- 172.21.15.175:0/3980539877 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f11b0014420 con 0x7f11c80f62a0 2023-12-06T03:27:31.115 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:31.115 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f11b402ffb0 msgr2=0x7f11b4032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f11b402ffb0 0x7f11b4032470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f11b80096a0 tx=0x7f11b8006b40).stop 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 msgr2=0x7f11c81064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c81064b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f11b0013510 tx=0x7f11b000b040).stop 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 shutdown_connections 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f11b402ffb0 0x7f11b4032470 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 --2- 172.21.15.175:0/3980539877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11c80f62a0 0x7f11c81064b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:31.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 >> 172.21.15.175:0/3980539877 conn(0x7f11c80f1aa0 msgr2=0x7f11c80f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:31.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 shutdown_connections 2023-12-06T03:27:31.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:31.115+0000 7f11c759e700 1 -- 172.21.15.175:0/3980539877 wait complete. 2023-12-06T03:27:31.117 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:30.461364+0000 mgr.smithi055.qjkkyb (mgr.14164) 190 : cluster [DBG] pgmap v178: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:31.114389+0000 mon.smithi055 (mon.0) 260 : audit [DBG] from='client.? 172.21.15.175:0/3980539877' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:32.540 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:32.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:33.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 -- 172.21.15.175:0/606177356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 msgr2=0x7fcee40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:33.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 --2- 172.21.15.175:0/606177356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee40f5830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fcecc009fb0 tx=0x7fcecc009b30).stop 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 -- 172.21.15.175:0/606177356 shutdown_connections 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 --2- 172.21.15.175:0/606177356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee40f5830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 -- 172.21.15.175:0/606177356 >> 172.21.15.175:0/606177356 conn(0x7fcee40f0b70 msgr2=0x7fcee40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 -- 172.21.15.175:0/606177356 shutdown_connections 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.714+0000 7fcee9e09700 1 -- 172.21.15.175:0/606177356 wait complete. 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.715+0000 7fcee9e09700 1 Processor -- start 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.715+0000 7fcee9e09700 1 -- start start 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.715+0000 7fcee9e09700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee4109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:33.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.715+0000 7fcee9e09700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee4060240 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee4109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee4109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51676/0 (socket says 172.21.15.175:51676) 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee37fe700 1 -- 172.21.15.175:0/2407108828 learned_addr learned my addr 172.21.15.175:0/2407108828 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee37fe700 1 -- 172.21.15.175:0/2407108828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc005040 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee37fe700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee4109430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fcecc014000 tx=0x7fcecc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcecc0095f0 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee4060440 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcecc007e70 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcecc014ea0 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.716+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee4109ae0 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.717+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fcecc0079c0 con 0x7fcee40f5410 2023-12-06T03:27:33.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.717+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee4045ad0 con 0x7fcee40f5410 2023-12-06T03:27:33.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.718+0000 7fcedbfff700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fced002ffb0 0x7fced0032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:33.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.718+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fcecc040940 con 0x7fcee40f5410 2023-12-06T03:27:33.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.718+0000 7fcee2ffd700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fced002ffb0 0x7fced0032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:33.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.718+0000 7fcee2ffd700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fced002ffb0 0x7fced0032470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fced40096a0 tx=0x7fced4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:33.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.722+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcecc00f950 con 0x7fcee40f5410 2023-12-06T03:27:33.930 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.929+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcee40fae60 con 0x7fcee40f5410 2023-12-06T03:27:33.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.929+0000 7fcedbfff700 1 -- 172.21.15.175:0/2407108828 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcecc010230 con 0x7fcee40f5410 2023-12-06T03:27:33.931 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:33.931 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:33.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fced002ffb0 msgr2=0x7fced0032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:33.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fced002ffb0 0x7fced0032470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fced40096a0 tx=0x7fced4006b40).stop 2023-12-06T03:27:33.932 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 msgr2=0x7fcee4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee4109430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fcecc014000 tx=0x7fcecc00b040).stop 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 shutdown_connections 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fced002ffb0 0x7fced0032470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 --2- 172.21.15.175:0/2407108828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcee40f5410 0x7fcee4109430 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 >> 172.21.15.175:0/2407108828 conn(0x7fcee40f0b70 msgr2=0x7fcee40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 shutdown_connections 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:33.931+0000 7fcee9e09700 1 -- 172.21.15.175:0/2407108828 wait complete. 2023-12-06T03:27:33.933 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:32.461633+0000 mgr.smithi055.qjkkyb (mgr.14164) 191 : cluster [DBG] pgmap v179: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:34.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:34 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:33.930655+0000 mon.smithi055 (mon.0) 261 : audit [DBG] from='client.? 172.21.15.175:0/2407108828' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:35.328 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:35.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:35.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:34.461921+0000 mgr.smithi055.qjkkyb (mgr.14164) 192 : cluster [DBG] pgmap v180: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 -- 172.21.15.175:0/2189025614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 msgr2=0x7f329c0f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 --2- 172.21.15.175:0/2189025614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c0f66c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f3298009fb0 tx=0x7f3298009b30).stop 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 -- 172.21.15.175:0/2189025614 shutdown_connections 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 --2- 172.21.15.175:0/2189025614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c0f66c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 -- 172.21.15.175:0/2189025614 >> 172.21.15.175:0/2189025614 conn(0x7f329c0f1aa0 msgr2=0x7f329c0f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 -- 172.21.15.175:0/2189025614 shutdown_connections 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 -- 172.21.15.175:0/2189025614 wait complete. 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 Processor -- start 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.537+0000 7f32a34d2700 1 -- start start 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a34d2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c1064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:36.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a34d2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c1069a0 con 0x7f329c0f62a0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a24d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c1064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a24d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c1064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53286/0 (socket says 172.21.15.175:53286) 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a24d0700 1 -- 172.21.15.175:0/54912462 learned_addr learned my addr 172.21.15.175:0/54912462 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a24d0700 1 -- 172.21.15.175:0/54912462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298005040 con 0x7f329c0f62a0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a24d0700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c1064b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f3298013510 tx=0x7f329800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3298009320 con 0x7f329c0f62a0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3298013f00 con 0x7f329c0f62a0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c106ba0 con 0x7f329c0f62a0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3298014dc0 con 0x7f329c0f62a0 2023-12-06T03:27:36.541 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.538+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c107030 con 0x7f329c0f62a0 2023-12-06T03:27:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.539+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f3298014420 con 0x7f329c0f62a0 2023-12-06T03:27:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.539+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c045ad0 con 0x7f329c0f62a0 2023-12-06T03:27:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.539+0000 7f3292ffd700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f328802ff60 0x7f3288032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.539+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f32980410f0 con 0x7f329c0f62a0 2023-12-06T03:27:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.540+0000 7f32a1ccf700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f328802ff60 0x7f3288032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.540+0000 7f32a1ccf700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f328802ff60 0x7f3288032420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f328c0096a0 tx=0x7f328c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:36.545 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.545+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f32980095f0 con 0x7f329c0f62a0 2023-12-06T03:27:36.749 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.748+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f329c107ee0 con 0x7f329c0f62a0 2023-12-06T03:27:36.750 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.749+0000 7f3292ffd700 1 -- 172.21.15.175:0/54912462 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f329801cb10 con 0x7f329c0f62a0 2023-12-06T03:27:36.750 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:36.750 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:36.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f328802ff60 msgr2=0x7f3288032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:36.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f328802ff60 0x7f3288032420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f328c0096a0 tx=0x7f328c006b40).stop 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 msgr2=0x7f329c1064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c1064b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f3298013510 tx=0x7f329800b040).stop 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 shutdown_connections 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f328802ff60 0x7f3288032420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 --2- 172.21.15.175:0/54912462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f329c0f62a0 0x7f329c1064b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 >> 172.21.15.175:0/54912462 conn(0x7f329c0f1aa0 msgr2=0x7f329c0f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 shutdown_connections 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:36.750+0000 7f32a34d2700 1 -- 172.21.15.175:0/54912462 wait complete. 2023-12-06T03:27:36.752 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:37.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:36.462210+0000 mgr.smithi055.qjkkyb (mgr.14164) 193 : cluster [DBG] pgmap v181: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:37.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:36.749971+0000 mon.smithi055 (mon.0) 262 : audit [DBG] from='client.? 172.21.15.175:0/54912462' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:38.122 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:38.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 -- 172.21.15.175:0/3124609147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 msgr2=0x7f795c0f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 --2- 172.21.15.175:0/3124609147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f90f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f794c009fb0 tx=0x7f794c009b30).stop 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 -- 172.21.15.175:0/3124609147 shutdown_connections 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 --2- 172.21.15.175:0/3124609147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f90f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 -- 172.21.15.175:0/3124609147 >> 172.21.15.175:0/3124609147 conn(0x7f795c0f0b70 msgr2=0x7f795c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 -- 172.21.15.175:0/3124609147 shutdown_connections 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 -- 172.21.15.175:0/3124609147 wait complete. 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.309+0000 7f79627b4700 1 Processor -- start 2023-12-06T03:27:39.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f79627b4700 1 -- start start 2023-12-06T03:27:39.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f79627b4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:39.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f79627b4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0f4ff0 con 0x7f795c060280 2023-12-06T03:27:39.312 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f795bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f795bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53296/0 (socket says 172.21.15.175:53296) 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f795bfff700 1 -- 172.21.15.175:0/2962550219 learned_addr learned my addr 172.21.15.175:0/2962550219 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.310+0000 7f795bfff700 1 -- 172.21.15.175:0/2962550219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c005040 con 0x7f795c060280 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.311+0000 7f795bfff700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f6b70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f794c009830 tx=0x7f794c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.311+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f794c009320 con 0x7f795c060280 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.311+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c0f5190 con 0x7f795c060280 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.311+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c0f5620 con 0x7f795c060280 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.311+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f794c0136c0 con 0x7f795c060280 2023-12-06T03:27:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.311+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f794c01ee90 con 0x7f795c060280 2023-12-06T03:27:39.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.312+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f794c014420 con 0x7f795c060280 2023-12-06T03:27:39.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.312+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c004fa0 con 0x7f795c060280 2023-12-06T03:27:39.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.312+0000 7f7958ff9700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f794402ff10 0x7f79440323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:39.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.312+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f794c041200 con 0x7f795c060280 2023-12-06T03:27:39.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.312+0000 7f795b7fe700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f794402ff10 0x7f79440323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:39.314 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.313+0000 7f795b7fe700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f794402ff10 0x7f79440323d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f79500096a0 tx=0x7f7950006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:39.318 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.318+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f794c01c500 con 0x7f795c060280 2023-12-06T03:27:39.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.521+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f793c005670 con 0x7f795c060280 2023-12-06T03:27:39.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.522+0000 7f7958ff9700 1 -- 172.21.15.175:0/2962550219 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f794c02c2d0 con 0x7f795c060280 2023-12-06T03:27:39.523 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:39.523 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.523+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f794402ff10 msgr2=0x7f79440323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.523+0000 7f79627b4700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f794402ff10 0x7f79440323d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f79500096a0 tx=0x7f7950006b40).stop 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.523+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 msgr2=0x7f795c0f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.523+0000 7f79627b4700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f6b70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f794c009830 tx=0x7f794c00b040).stop 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.524+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 shutdown_connections 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.524+0000 7f79627b4700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f794402ff10 0x7f79440323d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.524+0000 7f79627b4700 1 --2- 172.21.15.175:0/2962550219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f795c060280 0x7f795c0f6b70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:39.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.524+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 >> 172.21.15.175:0/2962550219 conn(0x7f795c0f0b70 msgr2=0x7f795c0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:39.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.524+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 shutdown_connections 2023-12-06T03:27:39.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:39.524+0000 7f79627b4700 1 -- 172.21.15.175:0/2962550219 wait complete. 2023-12-06T03:27:39.526 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:38.462579+0000 mgr.smithi055.qjkkyb (mgr.14164) 194 : cluster [DBG] pgmap v182: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:39.523231+0000 mon.smithi055 (mon.0) 263 : audit [DBG] from='client.? 172.21.15.175:0/2962550219' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:40.901 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:40.901 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:40.462904+0000 mgr.smithi055.qjkkyb (mgr.14164) 195 : cluster [DBG] pgmap v183: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:42.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 -- 172.21.15.175:0/3723226032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 msgr2=0x7efcfc0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:42.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 --2- 172.21.15.175:0/3723226032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0f5830 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7efce4009fb0 tx=0x7efce4009b30).stop 2023-12-06T03:27:42.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 -- 172.21.15.175:0/3723226032 shutdown_connections 2023-12-06T03:27:42.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 --2- 172.21.15.175:0/3723226032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0f5830 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:42.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 -- 172.21.15.175:0/3723226032 >> 172.21.15.175:0/3723226032 conn(0x7efcfc0f0b70 msgr2=0x7efcfc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:42.095 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 -- 172.21.15.175:0/3723226032 shutdown_connections 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.093+0000 7efd00925700 1 -- 172.21.15.175:0/3723226032 wait complete. 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efd00925700 1 Processor -- start 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efd00925700 1 -- start start 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efd00925700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efd00925700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0fb820 con 0x7efcfc0f5410 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efcfa59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efcfa59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53312/0 (socket says 172.21.15.175:53312) 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efcfa59c700 1 -- 172.21.15.175:0/2582642336 learned_addr learned my addr 172.21.15.175:0/2582642336 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.094+0000 7efcfa59c700 1 -- 172.21.15.175:0/2582642336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce4005040 con 0x7efcfc0f5410 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.095+0000 7efcfa59c700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0fd3a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7efce4013510 tx=0x7efce400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.095+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efce4009320 con 0x7efcfc0f5410 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.095+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc0fb9c0 con 0x7efcfc0f5410 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.095+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc0fbe50 con 0x7efcfc0f5410 2023-12-06T03:27:42.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.095+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efce4013f00 con 0x7efcfc0f5410 2023-12-06T03:27:42.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.095+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efce4014d90 con 0x7efcfc0f5410 2023-12-06T03:27:42.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.096+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7efce4014420 con 0x7efcfc0f5410 2023-12-06T03:27:42.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.096+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc059100 con 0x7efcfc0f5410 2023-12-06T03:27:42.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.096+0000 7efcf2ffd700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efce802ff60 0x7efce8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:42.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.096+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7efce4041130 con 0x7efcfc0f5410 2023-12-06T03:27:42.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.096+0000 7efcf9d9b700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efce802ff60 0x7efce8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:42.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.096+0000 7efcf9d9b700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efce802ff60 0x7efce8032420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7efcec0096a0 tx=0x7efcec006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:42.101 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.101+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efce40095f0 con 0x7efcfc0f5410 2023-12-06T03:27:42.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.304+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efcfc0fae60 con 0x7efcfc0f5410 2023-12-06T03:27:42.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.305+0000 7efcf2ffd700 1 -- 172.21.15.175:0/2582642336 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7efce401db70 con 0x7efcfc0f5410 2023-12-06T03:27:42.305 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:42.305 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:42.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.306+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efce802ff60 msgr2=0x7efce8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:42.307 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.306+0000 7efd00925700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efce802ff60 0x7efce8032420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7efcec0096a0 tx=0x7efcec006b40).stop 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 msgr2=0x7efcfc0fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0fd3a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7efce4013510 tx=0x7efce400b040).stop 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 shutdown_connections 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7efce802ff60 0x7efce8032420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 --2- 172.21.15.175:0/2582642336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efcfc0f5410 0x7efcfc0fd3a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 >> 172.21.15.175:0/2582642336 conn(0x7efcfc0f0b70 msgr2=0x7efcfc05fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 shutdown_connections 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:42.307+0000 7efd00925700 1 -- 172.21.15.175:0/2582642336 wait complete. 2023-12-06T03:27:42.308 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:42.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:42 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:42.305255+0000 mon.smithi055 (mon.0) 264 : audit [DBG] from='client.? 172.21.15.175:0/2582642336' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:43.654 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:43.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:42.463258+0000 mgr.smithi055.qjkkyb (mgr.14164) 196 : cluster [DBG] pgmap v184: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:44.851 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 -- 172.21.15.175:0/2304749454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 msgr2=0x7f7af00f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:44.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 --2- 172.21.15.175:0/2304749454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af00f55c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f7ae0009fb0 tx=0x7f7ae0009b30).stop 2023-12-06T03:27:44.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 -- 172.21.15.175:0/2304749454 shutdown_connections 2023-12-06T03:27:44.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 --2- 172.21.15.175:0/2304749454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af00f55c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:44.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 -- 172.21.15.175:0/2304749454 >> 172.21.15.175:0/2304749454 conn(0x7f7af00f09a0 msgr2=0x7f7af00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:44.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 -- 172.21.15.175:0/2304749454 shutdown_connections 2023-12-06T03:27:44.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.850+0000 7f7aeffff700 1 -- 172.21.15.175:0/2304749454 wait complete. 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeffff700 1 Processor -- start 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeffff700 1 -- start start 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af01064c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01069b0 con 0x7f7af00f51a0 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af01064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af01064c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:53326/0 (socket says 172.21.15.175:53326) 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeeffd700 1 -- 172.21.15.175:0/3080387224 learned_addr learned my addr 172.21.15.175:0/3080387224 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.851+0000 7f7aeeffd700 1 -- 172.21.15.175:0/3080387224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0005040 con 0x7f7af00f51a0 2023-12-06T03:27:44.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.852+0000 7f7aeeffd700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af01064c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f7ae0013510 tx=0x7f7ae000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.852+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae0009320 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.852+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af0106bb0 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.852+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ae0013f00 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.852+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae0014dc0 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.852+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af0107040 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.853+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af0045ad0 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.853+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7ae0014420 con 0x7f7af00f51a0 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.853+0000 7f7ad77fe700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7ad802ffb0 0x7f7ad8032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:44.855 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.853+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7ae0040fd0 con 0x7f7af00f51a0 2023-12-06T03:27:44.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.853+0000 7f7aee7fc700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7ad802ffb0 0x7f7ad8032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:44.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.854+0000 7f7aee7fc700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7ad802ffb0 0x7f7ad8032470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f7ae40096a0 tx=0x7f7ae4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:44.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:44.858+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7ae006d2b0 con 0x7f7af00f51a0 2023-12-06T03:27:45.064 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.063+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7af0107ef0 con 0x7f7af00f51a0 2023-12-06T03:27:45.065 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.064+0000 7f7ad77fe700 1 -- 172.21.15.175:0/3080387224 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7ae00095f0 con 0x7f7af00f51a0 2023-12-06T03:27:45.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:45.065 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7ad802ffb0 msgr2=0x7f7ad8032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7ad802ffb0 0x7f7ad8032470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f7ae40096a0 tx=0x7f7ae4006b40).stop 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 msgr2=0x7f7af01064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af01064c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f7ae0013510 tx=0x7f7ae000b040).stop 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 shutdown_connections 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7ad802ffb0 0x7f7ad8032470 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 --2- 172.21.15.175:0/3080387224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7af00f51a0 0x7f7af01064c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.065+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 >> 172.21.15.175:0/3080387224 conn(0x7f7af00f09a0 msgr2=0x7f7af00f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.066+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 shutdown_connections 2023-12-06T03:27:45.067 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:45.066+0000 7f7aeffff700 1 -- 172.21.15.175:0/3080387224 wait complete. 2023-12-06T03:27:45.068 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:44.463564+0000 mgr.smithi055.qjkkyb (mgr.14164) 197 : cluster [DBG] pgmap v185: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:46.446 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:46.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:46.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:45.064843+0000 mon.smithi055 (mon.0) 265 : audit [DBG] from='client.? 172.21.15.175:0/3080387224' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:47.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:46.463904+0000 mgr.smithi055.qjkkyb (mgr.14164) 198 : cluster [DBG] pgmap v186: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:47.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 -- 172.21.15.175:0/830367319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 msgr2=0x7f4b900f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:47.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 --2- 172.21.15.175:0/830367319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900f6e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f4b84009fb0 tx=0x7f4b84009b30).stop 2023-12-06T03:27:47.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 -- 172.21.15.175:0/830367319 shutdown_connections 2023-12-06T03:27:47.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 --2- 172.21.15.175:0/830367319 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900f6e20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:47.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 -- 172.21.15.175:0/830367319 >> 172.21.15.175:0/830367319 conn(0x7f4b900f0260 msgr2=0x7f4b900f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:47.621 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 -- 172.21.15.175:0/830367319 shutdown_connections 2023-12-06T03:27:47.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.619+0000 7f4b96d92700 1 -- 172.21.15.175:0/830367319 wait complete. 2023-12-06T03:27:47.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b96d92700 1 Processor -- start 2023-12-06T03:27:47.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b96d92700 1 -- start start 2023-12-06T03:27:47.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b96d92700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:47.622 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b96d92700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900fb8b0 con 0x7f4b900f6a00 2023-12-06T03:27:47.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b94b2e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:47.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b94b2e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:46762/0 (socket says 172.21.15.175:46762) 2023-12-06T03:27:47.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.620+0000 7f4b94b2e700 1 -- 172.21.15.175:0/170916084 learned_addr learned my addr 172.21.15.175:0/170916084 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b94b2e700 1 -- 172.21.15.175:0/170916084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84005040 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b94b2e700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900fd430 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f4b84013510 tx=0x7f4b8400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b84009320 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b900fba50 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b84013f00 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b84014dc0 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.621+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b900fbee0 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.622+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f4b840095f0 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.622+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b90059100 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.622+0000 7f4b8d7fa700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b8002ffb0 0x7f4b80032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.622+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4b84040ff0 con 0x7f4b900f6a00 2023-12-06T03:27:47.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.622+0000 7f4b8ffff700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b8002ffb0 0x7f4b80032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:47.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.623+0000 7f4b8ffff700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b8002ffb0 0x7f4b80032470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4b7c0096a0 tx=0x7f4b7c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:47.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.627+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4b8406d2b0 con 0x7f4b900f6a00 2023-12-06T03:27:47.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.834+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4b900faef0 con 0x7f4b900f6a00 2023-12-06T03:27:47.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.835+0000 7f4b8d7fa700 1 -- 172.21.15.175:0/170916084 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4b84014420 con 0x7f4b900f6a00 2023-12-06T03:27:47.836 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:47.836 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b8002ffb0 msgr2=0x7f4b80032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b8002ffb0 0x7f4b80032470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4b7c0096a0 tx=0x7f4b7c006b40).stop 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 msgr2=0x7f4b900fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900fd430 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f4b84013510 tx=0x7f4b8400b040).stop 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 shutdown_connections 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b8002ffb0 0x7f4b80032470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 --2- 172.21.15.175:0/170916084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b900f6a00 0x7f4b900fd430 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.836+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 >> 172.21.15.175:0/170916084 conn(0x7f4b900f0260 msgr2=0x7f4b900f4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.837+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 shutdown_connections 2023-12-06T03:27:47.838 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:47.837+0000 7f4b96d92700 1 -- 172.21.15.175:0/170916084 wait complete. 2023-12-06T03:27:47.839 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:48.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:48 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:47.836002+0000 mon.smithi055 (mon.0) 266 : audit [DBG] from='client.? 172.21.15.175:0/170916084' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:49.209 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:49.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:49.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:48.464268+0000 mgr.smithi055.qjkkyb (mgr.14164) 199 : cluster [DBG] pgmap v187: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.437+0000 7f15dbc82700 1 -- 172.21.15.175:0/1032132970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 msgr2=0x7f15d40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.437+0000 7f15dbc82700 1 --2- 172.21.15.175:0/1032132970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d40f5830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f15c8009fb0 tx=0x7f15c8009b30).stop 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.437+0000 7f15dbc82700 1 -- 172.21.15.175:0/1032132970 shutdown_connections 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.437+0000 7f15dbc82700 1 --2- 172.21.15.175:0/1032132970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d40f5830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 -- 172.21.15.175:0/1032132970 >> 172.21.15.175:0/1032132970 conn(0x7f15d40f0b70 msgr2=0x7f15d40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 -- 172.21.15.175:0/1032132970 shutdown_connections 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 -- 172.21.15.175:0/1032132970 wait complete. 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 Processor -- start 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 -- start start 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d4109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15dbc82700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4060240 con 0x7f15d40f5410 2023-12-06T03:27:50.440 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15d9a1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d4109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:50.441 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15d9a1e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d4109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:46772/0 (socket says 172.21.15.175:46772) 2023-12-06T03:27:50.441 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.438+0000 7f15d9a1e700 1 -- 172.21.15.175:0/1646029152 learned_addr learned my addr 172.21.15.175:0/1646029152 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:50.441 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15d9a1e700 1 -- 172.21.15.175:0/1646029152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8005040 con 0x7f15d40f5410 2023-12-06T03:27:50.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15d9a1e700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d4109430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f15c8013510 tx=0x7f15c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:50.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15c8009320 con 0x7f15d40f5410 2023-12-06T03:27:50.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d4060440 con 0x7f15d40f5410 2023-12-06T03:27:50.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d4109ae0 con 0x7f15d40f5410 2023-12-06T03:27:50.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15c8013f00 con 0x7f15d40f5410 2023-12-06T03:27:50.442 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.439+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15c8014cc0 con 0x7f15d40f5410 2023-12-06T03:27:50.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.440+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f15c8014420 con 0x7f15d40f5410 2023-12-06T03:27:50.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.440+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc004fa0 con 0x7f15d40f5410 2023-12-06T03:27:50.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.440+0000 7f15c67fc700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f15c002ff60 0x7f15c0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:50.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.440+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f15c8040fa0 con 0x7f15d40f5410 2023-12-06T03:27:50.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.441+0000 7f15d921d700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f15c002ff60 0x7f15c0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:50.443 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.441+0000 7f15d921d700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f15c002ff60 0x7f15c0032420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f15d00096a0 tx=0x7f15d0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:50.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.445+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f15c80095f0 con 0x7f15d40f5410 2023-12-06T03:27:50.650 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.648+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f15bc005670 con 0x7f15d40f5410 2023-12-06T03:27:50.651 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.649+0000 7f15c67fc700 1 -- 172.21.15.175:0/1646029152 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f15c8010e60 con 0x7f15d40f5410 2023-12-06T03:27:50.651 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:50.652 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:50.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.650+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f15c002ff60 msgr2=0x7f15c0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f15c002ff60 0x7f15c0032420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f15d00096a0 tx=0x7f15d0006b40).stop 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 msgr2=0x7f15d4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d4109430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f15c8013510 tx=0x7f15c800b040).stop 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 shutdown_connections 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f15c002ff60 0x7f15c0032420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 --2- 172.21.15.175:0/1646029152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f15d40f5410 0x7f15d4109430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 >> 172.21.15.175:0/1646029152 conn(0x7f15d40f0b70 msgr2=0x7f15d40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 shutdown_connections 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:50.651+0000 7f15dbc82700 1 -- 172.21.15.175:0/1646029152 wait complete. 2023-12-06T03:27:50.653 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:50.464507+0000 mgr.smithi055.qjkkyb (mgr.14164) 200 : cluster [DBG] pgmap v188: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:51.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:50.650237+0000 mon.smithi055 (mon.0) 267 : audit [DBG] from='client.? 172.21.15.175:0/1646029152' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:52.032 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:52.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:53.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- 172.21.15.175:0/1610002988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 msgr2=0x7f97880f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 --2- 172.21.15.175:0/1610002988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f97880f5830 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f9784009fb0 tx=0x7f9784009b30).stop 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- 172.21.15.175:0/1610002988 shutdown_connections 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 --2- 172.21.15.175:0/1610002988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f97880f5830 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- 172.21.15.175:0/1610002988 >> 172.21.15.175:0/1610002988 conn(0x7f97880f0b70 msgr2=0x7f97880f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- 172.21.15.175:0/1610002988 shutdown_connections 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- 172.21.15.175:0/1610002988 wait complete. 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 Processor -- start 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- start start 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f978810b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:53.177 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.174+0000 7f978ff89700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97881071e0 con 0x7f97880f5410 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978dd25700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f978810b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978dd25700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f978810b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:46790/0 (socket says 172.21.15.175:46790) 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978dd25700 1 -- 172.21.15.175:0/2466656378 learned_addr learned my addr 172.21.15.175:0/2466656378 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978dd25700 1 -- 172.21.15.175:0/2466656378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784005040 con 0x7f97880f5410 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978dd25700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f978810b7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9784014000 tx=0x7f978400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97840095f0 con 0x7f97880f5410 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9788060220 con 0x7f97880f5410 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9784007e70 con 0x7f97880f5410 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978810bce0 con 0x7f97880f5410 2023-12-06T03:27:53.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.175+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9784014ea0 con 0x7f97880f5410 2023-12-06T03:27:53.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.176+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9788045ad0 con 0x7f97880f5410 2023-12-06T03:27:53.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.176+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9784025070 con 0x7f97880f5410 2023-12-06T03:27:53.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.176+0000 7f977e7fc700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f977402ff60 0x7f9774032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:53.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.176+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9784040950 con 0x7f97880f5410 2023-12-06T03:27:53.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.177+0000 7f978d524700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f977402ff60 0x7f9774032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:53.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.177+0000 7f978d524700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f977402ff60 0x7f9774032420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f97780096a0 tx=0x7f9778006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:53.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.181+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f97840079c0 con 0x7f97880f5410 2023-12-06T03:27:53.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.390+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f978810c4d0 con 0x7f97880f5410 2023-12-06T03:27:53.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.390+0000 7f977e7fc700 1 -- 172.21.15.175:0/2466656378 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f978400f300 con 0x7f97880f5410 2023-12-06T03:27:53.392 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:53.392 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:53.393 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f977402ff60 msgr2=0x7f9774032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:53.393 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f977402ff60 0x7f9774032420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f97780096a0 tx=0x7f9778006b40).stop 2023-12-06T03:27:53.393 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 msgr2=0x7f978810b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:53.393 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f978810b7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9784014000 tx=0x7f978400b040).stop 2023-12-06T03:27:53.393 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 shutdown_connections 2023-12-06T03:27:53.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f977402ff60 0x7f9774032420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:53.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 --2- 172.21.15.175:0/2466656378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97880f5410 0x7f978810b7f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:53.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 >> 172.21.15.175:0/2466656378 conn(0x7f97880f0b70 msgr2=0x7f97880f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:53.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 shutdown_connections 2023-12-06T03:27:53.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:53.392+0000 7f978ff89700 1 -- 172.21.15.175:0/2466656378 wait complete. 2023-12-06T03:27:53.394 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:52.464868+0000 mgr.smithi055.qjkkyb (mgr.14164) 201 : cluster [DBG] pgmap v189: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:53.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:53.391518+0000 mon.smithi055 (mon.0) 268 : audit [DBG] from='client.? 172.21.15.175:0/2466656378' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:54.774 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:54.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:54.465331+0000 mgr.smithi055.qjkkyb (mgr.14164) 202 : cluster [DBG] pgmap v190: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:55.890 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 -- 172.21.15.175:0/2893309773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 msgr2=0x7f17400f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:55.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 --2- 172.21.15.175:0/2893309773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17400f4c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f1734009fb0 tx=0x7f1734009b30).stop 2023-12-06T03:27:55.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 -- 172.21.15.175:0/2893309773 shutdown_connections 2023-12-06T03:27:55.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 --2- 172.21.15.175:0/2893309773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17400f4c80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:55.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 -- 172.21.15.175:0/2893309773 >> 172.21.15.175:0/2893309773 conn(0x7f17400f02e0 msgr2=0x7f17400f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:55.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 -- 172.21.15.175:0/2893309773 shutdown_connections 2023-12-06T03:27:55.891 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.889+0000 7f17459b7700 1 -- 172.21.15.175:0/2893309773 wait complete. 2023-12-06T03:27:55.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f17459b7700 1 Processor -- start 2023-12-06T03:27:55.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f17459b7700 1 -- start start 2023-12-06T03:27:55.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f17459b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17401068e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:55.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f17459b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1740106dd0 con 0x7f17400f4860 2023-12-06T03:27:55.893 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f173effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17401068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f173effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17401068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59440/0 (socket says 172.21.15.175:59440) 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.890+0000 7f173effd700 1 -- 172.21.15.175:0/4088000592 learned_addr learned my addr 172.21.15.175:0/4088000592 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f173effd700 1 -- 172.21.15.175:0/4088000592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734005040 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f173effd700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17401068e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f1734013510 tx=0x7f173400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1734009320 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1740106fd0 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1734013f00 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1734014dc0 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.891+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1740107460 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.892+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f173401d3d0 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.892+0000 7f17277fe700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f172802ff60 0x7f1728032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.892+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f17340410e0 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.892+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1740045ad0 con 0x7f17400f4860 2023-12-06T03:27:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.892+0000 7f173e7fc700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f172802ff60 0x7f1728032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:55.895 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.893+0000 7f173e7fc700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f172802ff60 0x7f1728032420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f17300096a0 tx=0x7f1730006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:55.898 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:55.897+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1734012920 con 0x7f17400f4860 2023-12-06T03:27:56.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.106+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f174005f370 con 0x7f17400f4860 2023-12-06T03:27:56.107 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.106+0000 7f17277fe700 1 -- 172.21.15.175:0/4088000592 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f173402b020 con 0x7f17400f4860 2023-12-06T03:27:56.108 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:56.108 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f172802ff60 msgr2=0x7f1728032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f172802ff60 0x7f1728032420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f17300096a0 tx=0x7f1730006b40).stop 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 msgr2=0x7f17401068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17401068e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f1734013510 tx=0x7f173400b040).stop 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 shutdown_connections 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f172802ff60 0x7f1728032420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 --2- 172.21.15.175:0/4088000592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17400f4860 0x7f17401068e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:56.109 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 >> 172.21.15.175:0/4088000592 conn(0x7f17400f02e0 msgr2=0x7f17400f0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:56.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 shutdown_connections 2023-12-06T03:27:56.110 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:56.108+0000 7f17459b7700 1 -- 172.21.15.175:0/4088000592 wait complete. 2023-12-06T03:27:56.110 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:56.107462+0000 mon.smithi055 (mon.0) 269 : audit [DBG] from='client.? 172.21.15.175:0/4088000592' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:27:57.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:56.465634+0000 mgr.smithi055.qjkkyb (mgr.14164) 203 : cluster [DBG] pgmap v191: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:57.521 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:27:57.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 -- 172.21.15.175:0/1338585495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 msgr2=0x7f69940f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 --2- 172.21.15.175:0/1338585495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f69940f4be0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f6984009fb0 tx=0x7f6984009b30).stop 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 -- 172.21.15.175:0/1338585495 shutdown_connections 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 --2- 172.21.15.175:0/1338585495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f69940f4be0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 -- 172.21.15.175:0/1338585495 >> 172.21.15.175:0/1338585495 conn(0x7f69940f02e0 msgr2=0x7f69940f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 -- 172.21.15.175:0/1338585495 shutdown_connections 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.658+0000 7f6998d2c700 1 -- 172.21.15.175:0/1338585495 wait complete. 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f6998d2c700 1 Processor -- start 2023-12-06T03:27:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f6998d2c700 1 -- start start 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f6998d2c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f6994106700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f6998d2c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994106bf0 con 0x7f69940f47c0 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f699259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f6994106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f699259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f6994106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59452/0 (socket says 172.21.15.175:59452) 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.659+0000 7f699259c700 1 -- 172.21.15.175:0/753147095 learned_addr learned my addr 172.21.15.175:0/753147095 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f699259c700 1 -- 172.21.15.175:0/753147095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6984005040 con 0x7f69940f47c0 2023-12-06T03:27:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f699259c700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f6994106700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f6984013510 tx=0x7f698400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6984009320 con 0x7f69940f47c0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6994106df0 con 0x7f69940f47c0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6984013f00 con 0x7f69940f47c0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6984014d90 con 0x7f69940f47c0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.660+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6994107280 con 0x7f69940f47c0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.661+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6994045ad0 con 0x7f69940f47c0 2023-12-06T03:27:58.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.661+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f69840095f0 con 0x7f69940f47c0 2023-12-06T03:27:58.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.661+0000 7f6982ffd700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f697c02ff60 0x7f697c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:27:58.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.661+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6984041000 con 0x7f69940f47c0 2023-12-06T03:27:58.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.661+0000 7f6991d9b700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f697c02ff60 0x7f697c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:27:58.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.662+0000 7f6991d9b700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f697c02ff60 0x7f697c032420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f69880096a0 tx=0x7f6988006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:27:58.667 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.666+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6984014420 con 0x7f69940f47c0 2023-12-06T03:27:58.873 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.871+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6994108130 con 0x7f69940f47c0 2023-12-06T03:27:58.873 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.872+0000 7f6982ffd700 1 -- 172.21.15.175:0/753147095 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6984010230 con 0x7f69940f47c0 2023-12-06T03:27:58.874 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:27:58.874 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:27:58.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f697c02ff60 msgr2=0x7f697c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:58.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f697c02ff60 0x7f697c032420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f69880096a0 tx=0x7f6988006b40).stop 2023-12-06T03:27:58.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 msgr2=0x7f6994106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:27:58.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f6994106700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f6984013510 tx=0x7f698400b040).stop 2023-12-06T03:27:58.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 shutdown_connections 2023-12-06T03:27:58.875 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f697c02ff60 0x7f697c032420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:58.876 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 --2- 172.21.15.175:0/753147095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f69940f47c0 0x7f6994106700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:27:58.876 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 >> 172.21.15.175:0/753147095 conn(0x7f69940f02e0 msgr2=0x7f69940f0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:27:58.876 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 shutdown_connections 2023-12-06T03:27:58.876 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:27:58.874+0000 7f6998d2c700 1 -- 172.21.15.175:0/753147095 wait complete. 2023-12-06T03:27:58.876 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:27:59.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:27:58.465985+0000 mgr.smithi055.qjkkyb (mgr.14164) 204 : cluster [DBG] pgmap v192: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:27:59.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:27:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:27:58.873355+0000 mon.smithi055 (mon.0) 270 : audit [DBG] from='client.? 172.21.15.175:0/753147095' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:00.250 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:00.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 -- 172.21.15.175:0/3107726726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 msgr2=0x7f961c0f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 --2- 172.21.15.175:0/3107726726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c0f5750 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f960c009fb0 tx=0x7f960c009b30).stop 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 -- 172.21.15.175:0/3107726726 shutdown_connections 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 --2- 172.21.15.175:0/3107726726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c0f5750 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 -- 172.21.15.175:0/3107726726 >> 172.21.15.175:0/3107726726 conn(0x7f961c0f0a90 msgr2=0x7f961c0f2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 -- 172.21.15.175:0/3107726726 shutdown_connections 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 -- 172.21.15.175:0/3107726726 wait complete. 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.350+0000 7f9622981700 1 Processor -- start 2023-12-06T03:28:01.352 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.351+0000 7f9622981700 1 -- start start 2023-12-06T03:28:01.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.351+0000 7f9622981700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c106a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:01.353 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.351+0000 7f9622981700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c106f10 con 0x7f961c0f5330 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.351+0000 7f961bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.351+0000 7f961bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59460/0 (socket says 172.21.15.175:59460) 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.351+0000 7f961bfff700 1 -- 172.21.15.175:0/1774959184 learned_addr learned my addr 172.21.15.175:0/1774959184 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f961bfff700 1 -- 172.21.15.175:0/1774959184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c005040 con 0x7f961c0f5330 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f961bfff700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c106a20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f960c009f80 tx=0x7f960c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f960c009320 con 0x7f961c0f5330 2023-12-06T03:28:01.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c107110 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f960c013ec0 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f960c01ece0 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.352+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1075a0 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.353+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f960c01e420 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.353+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c045ad0 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.353+0000 7f9618ff9700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f960402ff60 0x7f9604032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.353+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f960c040e60 con 0x7f961c0f5330 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.353+0000 7f961b7fe700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f960402ff60 0x7f9604032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:01.355 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.353+0000 7f961b7fe700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f960402ff60 0x7f9604032420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f96100096a0 tx=0x7f9610006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:01.359 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.358+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f960c01e6f0 con 0x7f961c0f5330 2023-12-06T03:28:01.570 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.568+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f961c05ea50 con 0x7f961c0f5330 2023-12-06T03:28:01.570 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.569+0000 7f9618ff9700 1 -- 172.21.15.175:0/1774959184 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f960c02c2d0 con 0x7f961c0f5330 2023-12-06T03:28:01.571 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:01.571 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:01.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f960402ff60 msgr2=0x7f9604032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:01.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f960402ff60 0x7f9604032420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f96100096a0 tx=0x7f9610006b40).stop 2023-12-06T03:28:01.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 msgr2=0x7f961c106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:01.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c106a20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f960c009f80 tx=0x7f960c00b040).stop 2023-12-06T03:28:01.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 shutdown_connections 2023-12-06T03:28:01.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f960402ff60 0x7f9604032420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:01.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 --2- 172.21.15.175:0/1774959184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f961c0f5330 0x7f961c106a20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:01.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 >> 172.21.15.175:0/1774959184 conn(0x7f961c0f0a90 msgr2=0x7f961c0f18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:01.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 shutdown_connections 2023-12-06T03:28:01.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:01.571+0000 7f9622981700 1 -- 172.21.15.175:0/1774959184 wait complete. 2023-12-06T03:28:01.573 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:00.466377+0000 mgr.smithi055.qjkkyb (mgr.14164) 205 : cluster [DBG] pgmap v193: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:02.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:02 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:01.570170+0000 mon.smithi055 (mon.0) 271 : audit [DBG] from='client.? 172.21.15.175:0/1774959184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:02.965 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:02.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:02.466741+0000 mgr.smithi055.qjkkyb (mgr.14164) 206 : cluster [DBG] pgmap v194: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:04.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.108+0000 7fd49796c700 1 -- 172.21.15.175:0/1554027405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 msgr2=0x7fd4900f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:04.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.108+0000 7fd49796c700 1 --2- 172.21.15.175:0/1554027405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd4900f5830 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fd484009fb0 tx=0x7fd484009b30).stop 2023-12-06T03:28:04.111 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.108+0000 7fd49796c700 1 -- 172.21.15.175:0/1554027405 shutdown_connections 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.108+0000 7fd49796c700 1 --2- 172.21.15.175:0/1554027405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd4900f5830 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.108+0000 7fd49796c700 1 -- 172.21.15.175:0/1554027405 >> 172.21.15.175:0/1554027405 conn(0x7fd4900f0b70 msgr2=0x7fd4900f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.108+0000 7fd49796c700 1 -- 172.21.15.175:0/1554027405 shutdown_connections 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd49796c700 1 -- 172.21.15.175:0/1554027405 wait complete. 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd49796c700 1 Processor -- start 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd49796c700 1 -- start start 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd49796c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd490109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd49796c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490060220 con 0x7fd4900f5410 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd495708700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd490109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd495708700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd490109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:59472/0 (socket says 172.21.15.175:59472) 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.109+0000 7fd495708700 1 -- 172.21.15.175:0/771199558 learned_addr learned my addr 172.21.15.175:0/771199558 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd495708700 1 -- 172.21.15.175:0/771199558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484005040 con 0x7fd4900f5410 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd495708700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd490109450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fd484013510 tx=0x7fd48400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:04.112 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd484009320 con 0x7fd4900f5410 2023-12-06T03:28:04.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd490060420 con 0x7fd4900f5410 2023-12-06T03:28:04.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd484013f00 con 0x7fd4900f5410 2023-12-06T03:28:04.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd484014d90 con 0x7fd4900f5410 2023-12-06T03:28:04.113 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.110+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd490109b00 con 0x7fd4900f5410 2023-12-06T03:28:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.111+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd4840095f0 con 0x7fd4900f5410 2023-12-06T03:28:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.111+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd490059100 con 0x7fd4900f5410 2023-12-06T03:28:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.111+0000 7fd481ffb700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd47c02ffb0 0x7fd47c032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.111+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd484040fe0 con 0x7fd4900f5410 2023-12-06T03:28:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.111+0000 7fd494f07700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd47c02ffb0 0x7fd47c032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.112+0000 7fd494f07700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd47c02ffb0 0x7fd47c032470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd48c0096a0 tx=0x7fd48c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:04.117 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.116+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd48406d2b0 con 0x7fd4900f5410 2023-12-06T03:28:04.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.322+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd490045ad0 con 0x7fd4900f5410 2023-12-06T03:28:04.323 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.322+0000 7fd481ffb700 1 -- 172.21.15.175:0/771199558 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd484014420 con 0x7fd4900f5410 2023-12-06T03:28:04.324 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:04.324 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:04.325 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd47c02ffb0 msgr2=0x7fd47c032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd47c02ffb0 0x7fd47c032470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fd48c0096a0 tx=0x7fd48c006b40).stop 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 msgr2=0x7fd490109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd490109450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fd484013510 tx=0x7fd48400b040).stop 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 shutdown_connections 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd47c02ffb0 0x7fd47c032470 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 --2- 172.21.15.175:0/771199558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd4900f5410 0x7fd490109450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 >> 172.21.15.175:0/771199558 conn(0x7fd4900f0b70 msgr2=0x7fd4900f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 shutdown_connections 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:04.324+0000 7fd49796c700 1 -- 172.21.15.175:0/771199558 wait complete. 2023-12-06T03:28:04.326 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:04.323443+0000 mon.smithi055 (mon.0) 272 : audit [DBG] from='client.? 172.21.15.175:0/771199558' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:05.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:04.467079+0000 mgr.smithi055.qjkkyb (mgr.14164) 207 : cluster [DBG] pgmap v195: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:05.709 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:05.709 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:06.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 -- 172.21.15.175:0/3054630028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 msgr2=0x7f0c100f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:06.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 --2- 172.21.15.175:0/3054630028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c100f5830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f0c00009fb0 tx=0x7f0c00009b30).stop 2023-12-06T03:28:06.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 -- 172.21.15.175:0/3054630028 shutdown_connections 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 --2- 172.21.15.175:0/3054630028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c100f5830 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 -- 172.21.15.175:0/3054630028 >> 172.21.15.175:0/3054630028 conn(0x7f0c100f0b70 msgr2=0x7f0c100f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 -- 172.21.15.175:0/3054630028 shutdown_connections 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.899+0000 7f0c183cb700 1 -- 172.21.15.175:0/3054630028 wait complete. 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c183cb700 1 Processor -- start 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c183cb700 1 -- start start 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c183cb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c10106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c183cb700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10106c20 con 0x7f0c100f5410 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c16167700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c10106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c16167700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c10106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42328/0 (socket says 172.21.15.175:42328) 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.900+0000 7f0c16167700 1 -- 172.21.15.175:0/2586270639 learned_addr learned my addr 172.21.15.175:0/2586270639 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c16167700 1 -- 172.21.15.175:0/2586270639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00005040 con 0x7f0c100f5410 2023-12-06T03:28:06.903 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c16167700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c10106730 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f0c00013510 tx=0x7f0c0000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:06.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c00009320 con 0x7f0c100f5410 2023-12-06T03:28:06.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c10106e20 con 0x7f0c100f5410 2023-12-06T03:28:06.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c00013f00 con 0x7f0c100f5410 2023-12-06T03:28:06.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c00014d90 con 0x7f0c100f5410 2023-12-06T03:28:06.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.901+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101072b0 con 0x7f0c100f5410 2023-12-06T03:28:06.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.902+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0c000095f0 con 0x7f0c100f5410 2023-12-06T03:28:06.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.902+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c10045ad0 con 0x7f0c100f5410 2023-12-06T03:28:06.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.902+0000 7f0c06ffd700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0bfc02ff60 0x7f0bfc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:06.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.902+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0c00040fc0 con 0x7f0c100f5410 2023-12-06T03:28:06.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.902+0000 7f0c15966700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0bfc02ff60 0x7f0bfc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:06.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.902+0000 7f0c15966700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0bfc02ff60 0x7f0bfc032420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0c0c0096a0 tx=0x7f0c0c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:06.907 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:06.907+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0c00014420 con 0x7f0c100f5410 2023-12-06T03:28:07.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.115+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0c100fae60 con 0x7f0c100f5410 2023-12-06T03:28:07.116 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.115+0000 7f0c06ffd700 1 -- 172.21.15.175:0/2586270639 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0c00010230 con 0x7f0c100f5410 2023-12-06T03:28:07.117 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:07.117 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0bfc02ff60 msgr2=0x7f0bfc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0bfc02ff60 0x7f0bfc032420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0c0c0096a0 tx=0x7f0c0c006b40).stop 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 msgr2=0x7f0c10106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c10106730 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f0c00013510 tx=0x7f0c0000b040).stop 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 shutdown_connections 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0bfc02ff60 0x7f0bfc032420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 --2- 172.21.15.175:0/2586270639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c100f5410 0x7f0c10106730 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 >> 172.21.15.175:0/2586270639 conn(0x7f0c100f0b70 msgr2=0x7f0c100f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 shutdown_connections 2023-12-06T03:28:07.119 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:07.117+0000 7f0c183cb700 1 -- 172.21.15.175:0/2586270639 wait complete. 2023-12-06T03:28:07.120 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:06.467395+0000 mgr.smithi055.qjkkyb (mgr.14164) 208 : cluster [DBG] pgmap v196: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:07.116623+0000 mon.smithi055 (mon.0) 273 : audit [DBG] from='client.? 172.21.15.175:0/2586270639' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:08.492 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:08.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- 172.21.15.175:0/3032422778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 msgr2=0x7f2e280f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 --2- 172.21.15.175:0/3032422778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e280f5830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f2e10009fb0 tx=0x7f2e10009b30).stop 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- 172.21.15.175:0/3032422778 shutdown_connections 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 --2- 172.21.15.175:0/3032422778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e280f5830 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- 172.21.15.175:0/3032422778 >> 172.21.15.175:0/3032422778 conn(0x7f2e280f0b70 msgr2=0x7f2e280f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- 172.21.15.175:0/3032422778 shutdown_connections 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- 172.21.15.175:0/3032422778 wait complete. 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 Processor -- start 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- start start 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e281055d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.690+0000 7f2e2ca33700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28105ac0 con 0x7f2e280f5410 2023-12-06T03:28:09.692 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e281055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:09.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e281055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42340/0 (socket says 172.21.15.175:42340) 2023-12-06T03:28:09.693 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2659c700 1 -- 172.21.15.175:0/2437359265 learned_addr learned my addr 172.21.15.175:0/2437359265 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:09.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2659c700 1 -- 172.21.15.175:0/2437359265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10005040 con 0x7f2e280f5410 2023-12-06T03:28:09.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2659c700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e281055d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f2e10013510 tx=0x7f2e1000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:09.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e10009320 con 0x7f2e280f5410 2023-12-06T03:28:09.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e28105cc0 con 0x7f2e280f5410 2023-12-06T03:28:09.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e10013f00 con 0x7f2e280f5410 2023-12-06T03:28:09.694 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e10014d90 con 0x7f2e280f5410 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.691+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e28106150 con 0x7f2e280f5410 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.692+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2e100095f0 con 0x7f2e280f5410 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.692+0000 7f2e1effd700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e1402ff60 0x7f2e14032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.692+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2e100410c0 con 0x7f2e280f5410 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.692+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28059100 con 0x7f2e280f5410 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.692+0000 7f2e25d9b700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e1402ff60 0x7f2e14032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:09.695 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.693+0000 7f2e25d9b700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e1402ff60 0x7f2e14032420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f2e180096a0 tx=0x7f2e18006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:09.698 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.697+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2e10014420 con 0x7f2e280f5410 2023-12-06T03:28:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.901+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2e280fae60 con 0x7f2e280f5410 2023-12-06T03:28:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.901+0000 7f2e1effd700 1 -- 172.21.15.175:0/2437359265 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2e1002b020 con 0x7f2e280f5410 2023-12-06T03:28:09.903 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:09.903 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e1402ff60 msgr2=0x7f2e14032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e1402ff60 0x7f2e14032420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f2e180096a0 tx=0x7f2e18006b40).stop 2023-12-06T03:28:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 msgr2=0x7f2e281055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e281055d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f2e10013510 tx=0x7f2e1000b040).stop 2023-12-06T03:28:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 shutdown_connections 2023-12-06T03:28:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e1402ff60 0x7f2e14032420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 --2- 172.21.15.175:0/2437359265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e280f5410 0x7f2e281055d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 >> 172.21.15.175:0/2437359265 conn(0x7f2e280f0b70 msgr2=0x7f2e280f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 shutdown_connections 2023-12-06T03:28:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:09.903+0000 7f2e2ca33700 1 -- 172.21.15.175:0/2437359265 wait complete. 2023-12-06T03:28:09.905 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:08.467752+0000 mgr.smithi055.qjkkyb (mgr.14164) 209 : cluster [DBG] pgmap v197: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:10.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:10 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:09.902442+0000 mon.smithi055 (mon.0) 274 : audit [DBG] from='client.? 172.21.15.175:0/2437359265' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:11.283 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:11.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:10.468146+0000 mgr.smithi055.qjkkyb (mgr.14164) 210 : cluster [DBG] pgmap v198: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:12.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1546502666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 msgr2=0x7f4c680f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:12.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 --2- 172.21.15.175:0/1546502666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c680f4be0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f4c5c009fb0 tx=0x7f4c5c009b30).stop 2023-12-06T03:28:12.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1546502666 shutdown_connections 2023-12-06T03:28:12.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 --2- 172.21.15.175:0/1546502666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c680f4be0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:12.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1546502666 >> 172.21.15.175:0/1546502666 conn(0x7f4c680f02e0 msgr2=0x7f4c680f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:12.520 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1546502666 shutdown_connections 2023-12-06T03:28:12.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.518+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1546502666 wait complete. 2023-12-06T03:28:12.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.519+0000 7f4c6ccc8700 1 Processor -- start 2023-12-06T03:28:12.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.519+0000 7f4c6ccc8700 1 -- start start 2023-12-06T03:28:12.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.519+0000 7f4c6ccc8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c68106700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:12.521 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.519+0000 7f4c6ccc8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68106bf0 con 0x7f4c680f47c0 2023-12-06T03:28:12.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.519+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c68106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:12.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c68106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42360/0 (socket says 172.21.15.175:42360) 2023-12-06T03:28:12.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c6659c700 1 -- 172.21.15.175:0/1518694457 learned_addr learned my addr 172.21.15.175:0/1518694457 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:12.522 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c6659c700 1 -- 172.21.15.175:0/1518694457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c005040 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c6659c700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c68106700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f4c5c013510 tx=0x7f4c5c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c5c009320 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c68106df0 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c5c013f00 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c5c014d90 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.520+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68107280 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.521+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68045ad0 con 0x7f4c680f47c0 2023-12-06T03:28:12.523 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.521+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f4c5c0095f0 con 0x7f4c680f47c0 2023-12-06T03:28:12.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.521+0000 7f4c56ffd700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4c5002ff60 0x7f4c50032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:12.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.522+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4c5c041000 con 0x7f4c680f47c0 2023-12-06T03:28:12.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.522+0000 7f4c65d9b700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4c5002ff60 0x7f4c50032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:12.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.522+0000 7f4c65d9b700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4c5002ff60 0x7f4c50032420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4c580096a0 tx=0x7f4c58006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:12.527 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.526+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4c5c014420 con 0x7f4c680f47c0 2023-12-06T03:28:12.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.730+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4c68108130 con 0x7f4c680f47c0 2023-12-06T03:28:12.731 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.730+0000 7f4c56ffd700 1 -- 172.21.15.175:0/1518694457 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4c5c010230 con 0x7f4c680f47c0 2023-12-06T03:28:12.732 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:12.732 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4c5002ff60 msgr2=0x7f4c50032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4c5002ff60 0x7f4c50032420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4c580096a0 tx=0x7f4c58006b40).stop 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 msgr2=0x7f4c68106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c68106700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f4c5c013510 tx=0x7f4c5c00b040).stop 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 shutdown_connections 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4c5002ff60 0x7f4c50032420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 --2- 172.21.15.175:0/1518694457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c680f47c0 0x7f4c68106700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 >> 172.21.15.175:0/1518694457 conn(0x7f4c680f02e0 msgr2=0x7f4c680f0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 shutdown_connections 2023-12-06T03:28:12.733 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:12.732+0000 7f4c6ccc8700 1 -- 172.21.15.175:0/1518694457 wait complete. 2023-12-06T03:28:12.734 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:12.468454+0000 mgr.smithi055.qjkkyb (mgr.14164) 211 : cluster [DBG] pgmap v199: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:12.731346+0000 mon.smithi055 (mon.0) 275 : audit [DBG] from='client.? 172.21.15.175:0/1518694457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:14.105 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:14.105 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:15.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 -- 172.21.15.175:0/3246394566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e140f6510 msgr2=0x7f2e140f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:15.263 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 --2- 172.21.15.175:0/3246394566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e140f6510 0x7f2e140f6930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2dfc009fb0 tx=0x7f2dfc009b30).stop 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 -- 172.21.15.175:0/3246394566 shutdown_connections 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 --2- 172.21.15.175:0/3246394566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e140f6510 0x7f2e140f6930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 -- 172.21.15.175:0/3246394566 >> 172.21.15.175:0/3246394566 conn(0x7f2e140f1c70 msgr2=0x7f2e140f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 -- 172.21.15.175:0/3246394566 shutdown_connections 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.260+0000 7f2e19747700 1 -- 172.21.15.175:0/3246394566 wait complete. 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.261+0000 7f2e19747700 1 Processor -- start 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.261+0000 7f2e19747700 1 -- start start 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.261+0000 7f2e19747700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 0x7f2e14106c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.261+0000 7f2e19747700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14107120 con 0x7f2e14106810 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.261+0000 7f2e12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 0x7f2e14106c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e12ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 0x7f2e14106c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:42374/0 (socket says 172.21.15.175:42374) 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e12ffd700 1 -- 172.21.15.175:0/3657114236 learned_addr learned my addr 172.21.15.175:0/3657114236 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e12ffd700 1 -- 172.21.15.175:0/3657114236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc005040 con 0x7f2e14106810 2023-12-06T03:28:15.264 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e12ffd700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 0x7f2e14106c30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f2dfc009f80 tx=0x7f2dfc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:15.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dfc0136c0 con 0x7f2e14106810 2023-12-06T03:28:15.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14107320 con 0x7f2e14106810 2023-12-06T03:28:15.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2dfc013e70 con 0x7f2e14106810 2023-12-06T03:28:15.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dfc014420 con 0x7f2e14106810 2023-12-06T03:28:15.265 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.262+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e14107e70 con 0x7f2e14106810 2023-12-06T03:28:15.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.263+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2dfc025060 con 0x7f2e14106810 2023-12-06T03:28:15.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.263+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e14059100 con 0x7f2e14106810 2023-12-06T03:28:15.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.263+0000 7f2e0b7fe700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e0002ff60 0x7f2e00032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:15.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.263+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2dfc040f10 con 0x7f2e14106810 2023-12-06T03:28:15.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.263+0000 7f2e127fc700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e0002ff60 0x7f2e00032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:15.266 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.264+0000 7f2e127fc700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e0002ff60 0x7f2e00032420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f2e040096a0 tx=0x7f2e04006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:15.269 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.268+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2dfc01cb30 con 0x7f2e14106810 2023-12-06T03:28:15.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:14.468869+0000 mgr.smithi055.qjkkyb (mgr.14164) 212 : cluster [DBG] pgmap v200: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:15.476 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.475+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2e14045ad0 con 0x7f2e14106810 2023-12-06T03:28:15.476 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.475+0000 7f2e0b7fe700 1 -- 172.21.15.175:0/3657114236 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2dfc02d2d0 con 0x7f2e14106810 2023-12-06T03:28:15.477 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:15.477 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e0002ff60 msgr2=0x7f2e00032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e0002ff60 0x7f2e00032420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f2e040096a0 tx=0x7f2e04006b40).stop 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 msgr2=0x7f2e14106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 0x7f2e14106c30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f2dfc009f80 tx=0x7f2dfc009b30).stop 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 shutdown_connections 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2e0002ff60 0x7f2e00032420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:15.478 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 --2- 172.21.15.175:0/3657114236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e14106810 0x7f2e14106c30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:15.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 >> 172.21.15.175:0/3657114236 conn(0x7f2e140f1c70 msgr2=0x7f2e140f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:15.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 shutdown_connections 2023-12-06T03:28:15.479 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:15.477+0000 7f2e19747700 1 -- 172.21.15.175:0/3657114236 wait complete. 2023-12-06T03:28:15.479 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:16.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:16 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:15.476433+0000 mon.smithi055 (mon.0) 276 : audit [DBG] from='client.? 172.21.15.175:0/3657114236' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:16.861 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:16.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:17.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:16.469187+0000 mgr.smithi055.qjkkyb (mgr.14164) 213 : cluster [DBG] pgmap v201: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 -- 172.21.15.175:0/2596898092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 msgr2=0x7f6e180f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 --2- 172.21.15.175:0/2596898092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e180f90f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f6e08009fb0 tx=0x7f6e08009b30).stop 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 -- 172.21.15.175:0/2596898092 shutdown_connections 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 --2- 172.21.15.175:0/2596898092 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e180f90f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 -- 172.21.15.175:0/2596898092 >> 172.21.15.175:0/2596898092 conn(0x7f6e180f0b70 msgr2=0x7f6e180f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 -- 172.21.15.175:0/2596898092 shutdown_connections 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.017+0000 7f6e1d68e700 1 -- 172.21.15.175:0/2596898092 wait complete. 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e1d68e700 1 Processor -- start 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e1d68e700 1 -- start start 2023-12-06T03:28:18.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e1d68e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e181066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:18.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e1d68e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18106bd0 con 0x7f6e18060280 2023-12-06T03:28:18.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e181066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:18.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e181066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:46536/0 (socket says 172.21.15.175:46536) 2023-12-06T03:28:18.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.018+0000 7f6e16ffd700 1 -- 172.21.15.175:0/4155064168 learned_addr learned my addr 172.21.15.175:0/4155064168 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:18.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6e16ffd700 1 -- 172.21.15.175:0/4155064168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08005040 con 0x7f6e18060280 2023-12-06T03:28:18.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6e16ffd700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e181066e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f6e08013510 tx=0x7f6e0800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:18.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e08009320 con 0x7f6e18060280 2023-12-06T03:28:18.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e18106dd0 con 0x7f6e18060280 2023-12-06T03:28:18.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e08013f00 con 0x7f6e18060280 2023-12-06T03:28:18.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e08014dc0 con 0x7f6e18060280 2023-12-06T03:28:18.022 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.019+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e18107260 con 0x7f6e18060280 2023-12-06T03:28:18.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.020+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6e080095f0 con 0x7f6e18060280 2023-12-06T03:28:18.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.020+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e18045ad0 con 0x7f6e18060280 2023-12-06T03:28:18.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.020+0000 7f6dff7fe700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6e0002ffb0 0x7f6e00032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:18.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.020+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f6e080410c0 con 0x7f6e18060280 2023-12-06T03:28:18.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.020+0000 7f6e167fc700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6e0002ffb0 0x7f6e00032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:18.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.021+0000 7f6e167fc700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6e0002ffb0 0x7f6e00032470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6e0c0096a0 tx=0x7f6e0c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:18.026 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.025+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6e0801c790 con 0x7f6e18060280 2023-12-06T03:28:18.231 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.230+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6e18108110 con 0x7f6e18060280 2023-12-06T03:28:18.232 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.231+0000 7f6dff7fe700 1 -- 172.21.15.175:0/4155064168 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f6e08010230 con 0x7f6e18060280 2023-12-06T03:28:18.232 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:18.232 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6e0002ffb0 msgr2=0x7f6e00032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6e0002ffb0 0x7f6e00032470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f6e0c0096a0 tx=0x7f6e0c006b40).stop 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 msgr2=0x7f6e181066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e181066e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f6e08013510 tx=0x7f6e0800b040).stop 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 shutdown_connections 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f6e0002ffb0 0x7f6e00032470 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 --2- 172.21.15.175:0/4155064168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e18060280 0x7f6e181066e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.232+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 >> 172.21.15.175:0/4155064168 conn(0x7f6e180f0b70 msgr2=0x7f6e180f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.233+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 shutdown_connections 2023-12-06T03:28:18.234 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:18.233+0000 7f6e1d68e700 1 -- 172.21.15.175:0/4155064168 wait complete. 2023-12-06T03:28:18.235 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:18.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:18 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:18.232020+0000 mon.smithi055 (mon.0) 277 : audit [DBG] from='client.? 172.21.15.175:0/4155064168' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:19.647 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:19.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:19.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:18.469545+0000 mgr.smithi055.qjkkyb (mgr.14164) 214 : cluster [DBG] pgmap v202: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:20.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 -- 172.21.15.175:0/1413201225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 msgr2=0x7f315c0f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:20.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 --2- 172.21.15.175:0/1413201225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c0f8fa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f3158009fb0 tx=0x7f3158009b30).stop 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 -- 172.21.15.175:0/1413201225 shutdown_connections 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 --2- 172.21.15.175:0/1413201225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c0f8fa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 -- 172.21.15.175:0/1413201225 >> 172.21.15.175:0/1413201225 conn(0x7f315c0f09a0 msgr2=0x7f315c0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 -- 172.21.15.175:0/1413201225 shutdown_connections 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.843+0000 7f316285d700 1 -- 172.21.15.175:0/1413201225 wait complete. 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316285d700 1 Processor -- start 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316285d700 1 -- start start 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316285d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c106530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316285d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c106a20 con 0x7f315c060160 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316185b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c106530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316185b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c106530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:46558/0 (socket says 172.21.15.175:46558) 2023-12-06T03:28:20.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.844+0000 7f316185b700 1 -- 172.21.15.175:0/256992507 learned_addr learned my addr 172.21.15.175:0/256992507 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f316185b700 1 -- 172.21.15.175:0/256992507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158005040 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f316185b700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c106530 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f3158013510 tx=0x7f315800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3158009320 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315c106c20 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3158013f00 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3158014d90 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.845+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315c1070b0 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.846+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f31580263a0 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.846+0000 7f31527fc700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f314802ff60 0x7f3148032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.846+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3158041100 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.846+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315c045ad0 con 0x7f315c060160 2023-12-06T03:28:20.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.846+0000 7f316105a700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f314802ff60 0x7f3148032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:20.849 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.847+0000 7f316105a700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f314802ff60 0x7f3148032420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f314c0096a0 tx=0x7f314c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:20.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:20.851+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3158012920 con 0x7f315c060160 2023-12-06T03:28:21.056 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.055+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f315c107dd0 con 0x7f315c060160 2023-12-06T03:28:21.056 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.056+0000 7f31527fc700 1 -- 172.21.15.175:0/256992507 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f315802b020 con 0x7f315c060160 2023-12-06T03:28:21.057 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:21.057 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f314802ff60 msgr2=0x7f3148032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f314802ff60 0x7f3148032420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f314c0096a0 tx=0x7f314c006b40).stop 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 msgr2=0x7f315c106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c106530 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f3158013510 tx=0x7f315800b040).stop 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 shutdown_connections 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f314802ff60 0x7f3148032420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 --2- 172.21.15.175:0/256992507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f315c060160 0x7f315c106530 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 >> 172.21.15.175:0/256992507 conn(0x7f315c0f09a0 msgr2=0x7f315c0f14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 shutdown_connections 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:21.057+0000 7f316285d700 1 -- 172.21.15.175:0/256992507 wait complete. 2023-12-06T03:28:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:20.469918+0000 mgr.smithi055.qjkkyb (mgr.14164) 215 : cluster [DBG] pgmap v203: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:21.056676+0000 mon.smithi055 (mon.0) 278 : audit [DBG] from='client.? 172.21.15.175:0/256992507' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:22.447 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:22.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 -- 172.21.15.175:0/1896392424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 msgr2=0x7f57980f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 --2- 172.21.15.175:0/1896392424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f57980f5830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f5788009fb0 tx=0x7f5788009b30).stop 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 -- 172.21.15.175:0/1896392424 shutdown_connections 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 --2- 172.21.15.175:0/1896392424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f57980f5830 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 -- 172.21.15.175:0/1896392424 >> 172.21.15.175:0/1896392424 conn(0x7f57980f0b70 msgr2=0x7f57980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 -- 172.21.15.175:0/1896392424 shutdown_connections 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 -- 172.21.15.175:0/1896392424 wait complete. 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 Processor -- start 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.608+0000 7f579f981700 1 -- start start 2023-12-06T03:28:23.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579f981700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f5798109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579f981700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5798060240 con 0x7f57980f5410 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579d71d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f5798109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579d71d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f5798109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:46578/0 (socket says 172.21.15.175:46578) 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579d71d700 1 -- 172.21.15.175:0/2171338175 learned_addr learned my addr 172.21.15.175:0/2171338175 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579d71d700 1 -- 172.21.15.175:0/2171338175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788005040 con 0x7f57980f5410 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579d71d700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f5798109430 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f5788013510 tx=0x7f578800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5788009320 con 0x7f57980f5410 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5798060440 con 0x7f57980f5410 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5788013f00 con 0x7f57980f5410 2023-12-06T03:28:23.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.609+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5788014d90 con 0x7f57980f5410 2023-12-06T03:28:23.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.610+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5798109ae0 con 0x7f57980f5410 2023-12-06T03:28:23.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.610+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5798045ad0 con 0x7f57980f5410 2023-12-06T03:28:23.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.610+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f57880095f0 con 0x7f57980f5410 2023-12-06T03:28:23.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.611+0000 7f578dffb700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578402ff60 0x7f5784032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:23.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.611+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5788041000 con 0x7f57980f5410 2023-12-06T03:28:23.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.611+0000 7f579cf1c700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578402ff60 0x7f5784032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:23.614 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.611+0000 7f579cf1c700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578402ff60 0x7f5784032420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f57940096a0 tx=0x7f5794006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:23.616 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.615+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f578801d7f0 con 0x7f57980f5410 2023-12-06T03:28:23.822 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.821+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f57980fae60 con 0x7f57980f5410 2023-12-06T03:28:23.823 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.822+0000 7f578dffb700 1 -- 172.21.15.175:0/2171338175 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5788010230 con 0x7f57980f5410 2023-12-06T03:28:23.823 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:23.823 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:23.824 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578402ff60 msgr2=0x7f5784032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578402ff60 0x7f5784032420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f57940096a0 tx=0x7f5794006b40).stop 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 msgr2=0x7f5798109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f5798109430 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f5788013510 tx=0x7f578800b040).stop 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 shutdown_connections 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f578402ff60 0x7f5784032420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 --2- 172.21.15.175:0/2171338175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57980f5410 0x7f5798109430 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 >> 172.21.15.175:0/2171338175 conn(0x7f57980f0b70 msgr2=0x7f57980f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 shutdown_connections 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:23.823+0000 7f579f981700 1 -- 172.21.15.175:0/2171338175 wait complete. 2023-12-06T03:28:23.825 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:22.470234+0000 mgr.smithi055.qjkkyb (mgr.14164) 216 : cluster [DBG] pgmap v204: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:22.555757+0000 mon.smithi055 (mon.0) 279 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:28:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:22.558940+0000 mon.smithi055 (mon.0) 280 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:28:24.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:24 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:23.822879+0000 mon.smithi055 (mon.0) 281 : audit [DBG] from='client.? 172.21.15.175:0/2171338175' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:25.211 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:25.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:25.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:24.470505+0000 mgr.smithi055.qjkkyb (mgr.14164) 217 : cluster [DBG] pgmap v205: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 -- 172.21.15.175:0/3047501219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 msgr2=0x7f08200f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 --2- 172.21.15.175:0/3047501219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f08200f5830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f0814009fb0 tx=0x7f0814009b30).stop 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 -- 172.21.15.175:0/3047501219 shutdown_connections 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 --2- 172.21.15.175:0/3047501219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f08200f5830 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 -- 172.21.15.175:0/3047501219 >> 172.21.15.175:0/3047501219 conn(0x7f08200f0b70 msgr2=0x7f08200f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 -- 172.21.15.175:0/3047501219 shutdown_connections 2023-12-06T03:28:26.327 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 -- 172.21.15.175:0/3047501219 wait complete. 2023-12-06T03:28:26.328 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.325+0000 7f08271ee700 1 Processor -- start 2023-12-06T03:28:26.328 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.326+0000 7f08271ee700 1 -- start start 2023-12-06T03:28:26.328 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.326+0000 7f08271ee700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f0820106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:26.328 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.326+0000 7f08271ee700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0820106c20 con 0x7f08200f5410 2023-12-06T03:28:26.329 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.326+0000 7f0824f8a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f0820106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.326+0000 7f0824f8a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f0820106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33272/0 (socket says 172.21.15.175:33272) 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.326+0000 7f0824f8a700 1 -- 172.21.15.175:0/1192635954 learned_addr learned my addr 172.21.15.175:0/1192635954 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f0824f8a700 1 -- 172.21.15.175:0/1192635954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814005040 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f0824f8a700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f0820106730 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0814013510 tx=0x7f081400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0814009320 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0820106e20 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0814013f00 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0814014d90 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.327+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08201072b0 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.328+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f08140095f0 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.328+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0820045ad0 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.328+0000 7f081d7fa700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f081002ff60 0x7f0810032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.328+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0814041000 con 0x7f08200f5410 2023-12-06T03:28:26.330 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.328+0000 7f081ffff700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f081002ff60 0x7f0810032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:26.331 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.329+0000 7f081ffff700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f081002ff60 0x7f0810032420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f080c0096a0 tx=0x7f080c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:26.334 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.333+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0814014420 con 0x7f08200f5410 2023-12-06T03:28:26.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.539+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f08200fbf90 con 0x7f08200f5410 2023-12-06T03:28:26.540 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.539+0000 7f081d7fa700 1 -- 172.21.15.175:0/1192635954 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0814010230 con 0x7f08200f5410 2023-12-06T03:28:26.540 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:26.540 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f081002ff60 msgr2=0x7f0810032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f081002ff60 0x7f0810032420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f080c0096a0 tx=0x7f080c006b40).stop 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 msgr2=0x7f0820106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f0820106730 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f0814013510 tx=0x7f081400b040).stop 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 shutdown_connections 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f081002ff60 0x7f0810032420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:26.542 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 --2- 172.21.15.175:0/1192635954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08200f5410 0x7f0820106730 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:26.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 >> 172.21.15.175:0/1192635954 conn(0x7f08200f0b70 msgr2=0x7f08200f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:26.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 shutdown_connections 2023-12-06T03:28:26.543 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:26.541+0000 7f08271ee700 1 -- 172.21.15.175:0/1192635954 wait complete. 2023-12-06T03:28:26.543 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:27.935 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:27.936 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:26.470848+0000 mgr.smithi055.qjkkyb (mgr.14164) 218 : cluster [DBG] pgmap v206: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:27.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:26.540411+0000 mon.smithi055 (mon.0) 282 : audit [DBG] from='client.? 172.21.15.175:0/1192635954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:29.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 -- 172.21.15.175:0/424404872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 msgr2=0x7fa7540f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 --2- 172.21.15.175:0/424404872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa7540f6930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fa748009fb0 tx=0x7fa748009b30).stop 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 -- 172.21.15.175:0/424404872 shutdown_connections 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 --2- 172.21.15.175:0/424404872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa7540f6930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 -- 172.21.15.175:0/424404872 >> 172.21.15.175:0/424404872 conn(0x7fa7540f1c70 msgr2=0x7fa7540f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 -- 172.21.15.175:0/424404872 shutdown_connections 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.170+0000 7fa75b4dd700 1 -- 172.21.15.175:0/424404872 wait complete. 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa75b4dd700 1 Processor -- start 2023-12-06T03:28:29.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa75b4dd700 1 -- start start 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa75b4dd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa754106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa75b4dd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa754106c20 con 0x7fa7540f6510 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa759279700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa754106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa759279700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa754106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33288/0 (socket says 172.21.15.175:33288) 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.171+0000 7fa759279700 1 -- 172.21.15.175:0/581270621 learned_addr learned my addr 172.21.15.175:0/581270621 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa759279700 1 -- 172.21.15.175:0/581270621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa748005040 con 0x7fa7540f6510 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa759279700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa754106730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fa748013510 tx=0x7fa74800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa748009320 con 0x7fa7540f6510 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa754106e20 con 0x7fa7540f6510 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa748013f00 con 0x7fa7540f6510 2023-12-06T03:28:29.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa748014d90 con 0x7fa7540f6510 2023-12-06T03:28:29.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.172+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7541072b0 con 0x7fa7540f6510 2023-12-06T03:28:29.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.173+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa7480095f0 con 0x7fa7540f6510 2023-12-06T03:28:29.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.173+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa754045ad0 con 0x7fa7540f6510 2023-12-06T03:28:29.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.173+0000 7fa745ffb700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa74002ff60 0x7fa740032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:29.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.173+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa748040fc0 con 0x7fa7540f6510 2023-12-06T03:28:29.180 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.174+0000 7fa758a78700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa74002ff60 0x7fa740032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.174+0000 7fa758a78700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa74002ff60 0x7fa740032420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fa7500096a0 tx=0x7fa750006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:29.181 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.178+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa748014420 con 0x7fa7540f6510 2023-12-06T03:28:29.383 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.382+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa7540fbf60 con 0x7fa7540f6510 2023-12-06T03:28:29.384 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.383+0000 7fa745ffb700 1 -- 172.21.15.175:0/581270621 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fa748010230 con 0x7fa7540f6510 2023-12-06T03:28:29.386 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:29.386 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa74002ff60 msgr2=0x7fa740032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa74002ff60 0x7fa740032420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fa7500096a0 tx=0x7fa750006b40).stop 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 msgr2=0x7fa754106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa754106730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fa748013510 tx=0x7fa74800b040).stop 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 shutdown_connections 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fa74002ff60 0x7fa740032420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 --2- 172.21.15.175:0/581270621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7540f6510 0x7fa754106730 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 >> 172.21.15.175:0/581270621 conn(0x7fa7540f1c70 msgr2=0x7fa7540f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:29.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 shutdown_connections 2023-12-06T03:28:29.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:29.384+0000 7fa75b4dd700 1 -- 172.21.15.175:0/581270621 wait complete. 2023-12-06T03:28:29.388 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:28.471209+0000 mgr.smithi055.qjkkyb (mgr.14164) 219 : cluster [DBG] pgmap v207: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:29.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:29.383778+0000 mon.smithi055 (mon.0) 283 : audit [DBG] from='client.? 172.21.15.175:0/581270621' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:30.786 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:30.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:31.940 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 -- 172.21.15.175:0/1022317414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 msgr2=0x7f94ec0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 --2- 172.21.15.175:0/1022317414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec0f6930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f94e0009fb0 tx=0x7f94e0009b30).stop 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 -- 172.21.15.175:0/1022317414 shutdown_connections 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 --2- 172.21.15.175:0/1022317414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec0f6930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 -- 172.21.15.175:0/1022317414 >> 172.21.15.175:0/1022317414 conn(0x7f94ec0f1c70 msgr2=0x7f94ec0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 -- 172.21.15.175:0/1022317414 shutdown_connections 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.937+0000 7f94f34af700 1 -- 172.21.15.175:0/1022317414 wait complete. 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.938+0000 7f94f34af700 1 Processor -- start 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.938+0000 7f94f34af700 1 -- start start 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.938+0000 7f94f34af700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.938+0000 7f94f34af700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec106c20 con 0x7f94ec0f6510 2023-12-06T03:28:31.941 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f124b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f124b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33298/0 (socket says 172.21.15.175:33298) 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f124b700 1 -- 172.21.15.175:0/3033511054 learned_addr learned my addr 172.21.15.175:0/3033511054 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f124b700 1 -- 172.21.15.175:0/3033511054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0005040 con 0x7f94ec0f6510 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f124b700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec106730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f94e0013510 tx=0x7f94e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94e0009320 con 0x7f94ec0f6510 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec106e20 con 0x7f94ec0f6510 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94e0013f00 con 0x7f94ec0f6510 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94e0014d90 con 0x7f94ec0f6510 2023-12-06T03:28:31.942 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.939+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec1072b0 con 0x7f94ec0f6510 2023-12-06T03:28:31.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.940+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f94e00095f0 con 0x7f94ec0f6510 2023-12-06T03:28:31.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.940+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec045ad0 con 0x7f94ec0f6510 2023-12-06T03:28:31.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.940+0000 7f94ddffb700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f94d802ff60 0x7f94d8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:31.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.940+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f94e0041000 con 0x7f94ec0f6510 2023-12-06T03:28:31.943 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.941+0000 7f94f0a4a700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f94d802ff60 0x7f94d8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.941+0000 7f94f0a4a700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f94d802ff60 0x7f94d8032420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f94e80096a0 tx=0x7f94e8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:31.946 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:31.945+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f94e0014420 con 0x7f94ec0f6510 2023-12-06T03:28:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:30.471553+0000 mgr.smithi055.qjkkyb (mgr.14164) 220 : cluster [DBG] pgmap v208: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:32.153 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.151+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f94ec0fbf60 con 0x7f94ec0f6510 2023-12-06T03:28:32.153 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.152+0000 7f94ddffb700 1 -- 172.21.15.175:0/3033511054 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f94e001d7f0 con 0x7f94ec0f6510 2023-12-06T03:28:32.153 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:32.154 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f94d802ff60 msgr2=0x7f94d8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f94d802ff60 0x7f94d8032420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f94e80096a0 tx=0x7f94e8006b40).stop 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 msgr2=0x7f94ec106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec106730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f94e0013510 tx=0x7f94e000b040).stop 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 shutdown_connections 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f94d802ff60 0x7f94d8032420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 --2- 172.21.15.175:0/3033511054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f94ec0f6510 0x7f94ec106730 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 >> 172.21.15.175:0/3033511054 conn(0x7f94ec0f1c70 msgr2=0x7f94ec0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:32.155 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 shutdown_connections 2023-12-06T03:28:32.156 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:32.154+0000 7f94f34af700 1 -- 172.21.15.175:0/3033511054 wait complete. 2023-12-06T03:28:32.156 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:32.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:32 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:32.153293+0000 mon.smithi055 (mon.0) 284 : audit [DBG] from='client.? 172.21.15.175:0/3033511054' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:33.547 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:33.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:32.471924+0000 mgr.smithi055.qjkkyb (mgr.14164) 221 : cluster [DBG] pgmap v209: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 -- 172.21.15.175:0/3675729401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 msgr2=0x7f9f080f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 --2- 172.21.15.175:0/3675729401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f080f6930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f9ef8009fb0 tx=0x7f9ef8009b30).stop 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 -- 172.21.15.175:0/3675729401 shutdown_connections 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 --2- 172.21.15.175:0/3675729401 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f080f6930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 -- 172.21.15.175:0/3675729401 >> 172.21.15.175:0/3675729401 conn(0x7f9f080f1c70 msgr2=0x7f9f080f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 -- 172.21.15.175:0/3675729401 shutdown_connections 2023-12-06T03:28:34.716 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.714+0000 7f9f0f70f700 1 -- 172.21.15.175:0/3675729401 wait complete. 2023-12-06T03:28:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0f70f700 1 Processor -- start 2023-12-06T03:28:34.717 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0f70f700 1 -- start start 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0f70f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f08106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0f70f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08106c20 con 0x7f9f080f6510 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0d4ab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f08106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0d4ab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f08106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:33314/0 (socket says 172.21.15.175:33314) 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.715+0000 7f9f0d4ab700 1 -- 172.21.15.175:0/70229053 learned_addr learned my addr 172.21.15.175:0/70229053 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9f0d4ab700 1 -- 172.21.15.175:0/70229053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8005040 con 0x7f9f080f6510 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9f0d4ab700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f08106730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9ef8014000 tx=0x7f9ef800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ef80095f0 con 0x7f9f080f6510 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f08106e20 con 0x7f9f080f6510 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ef8007e70 con 0x7f9f080f6510 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ef8014ea0 con 0x7f9f080f6510 2023-12-06T03:28:34.718 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.716+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f081072b0 con 0x7f9f080f6510 2023-12-06T03:28:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.717+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9ef8025070 con 0x7f9f080f6510 2023-12-06T03:28:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.717+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f08045ad0 con 0x7f9f080f6510 2023-12-06T03:28:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.717+0000 7f9efdffb700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9ef402ff60 0x7f9ef4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:34.719 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.717+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9ef8040a30 con 0x7f9f080f6510 2023-12-06T03:28:34.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.717+0000 7f9f0ccaa700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9ef402ff60 0x7f9ef4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:34.720 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.718+0000 7f9f0ccaa700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9ef402ff60 0x7f9ef4032420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f9f040096a0 tx=0x7f9f04006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:34.723 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.722+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9ef80079c0 con 0x7f9f080f6510 2023-12-06T03:28:34.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.930+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9f080fbf60 con 0x7f9f080f6510 2023-12-06T03:28:34.931 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.930+0000 7f9efdffb700 1 -- 172.21.15.175:0/70229053 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9ef800f300 con 0x7f9f080f6510 2023-12-06T03:28:34.932 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:34.932 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:34.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9ef402ff60 msgr2=0x7f9ef4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:34.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9ef402ff60 0x7f9ef4032420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f9f040096a0 tx=0x7f9f04006b40).stop 2023-12-06T03:28:34.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 msgr2=0x7f9f08106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:34.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f08106730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9ef8014000 tx=0x7f9ef800b040).stop 2023-12-06T03:28:34.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 shutdown_connections 2023-12-06T03:28:34.933 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9ef402ff60 0x7f9ef4032420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:34.934 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 --2- 172.21.15.175:0/70229053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f080f6510 0x7f9f08106730 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:34.934 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 >> 172.21.15.175:0/70229053 conn(0x7f9f080f1c70 msgr2=0x7f9f080f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:34.934 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 shutdown_connections 2023-12-06T03:28:34.934 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:34.932+0000 7f9f0f70f700 1 -- 172.21.15.175:0/70229053 wait complete. 2023-12-06T03:28:34.934 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:34.472193+0000 mgr.smithi055.qjkkyb (mgr.14164) 222 : cluster [DBG] pgmap v210: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:34.931574+0000 mon.smithi055 (mon.0) 285 : audit [DBG] from='client.? 172.21.15.175:0/70229053' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:36.307 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:36.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 -- 172.21.15.175:0/2030857809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 msgr2=0x7f47300f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 --2- 172.21.15.175:0/2030857809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f47300f6d10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f472c009fb0 tx=0x7f472c009b30).stop 2023-12-06T03:28:37.512 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 -- 172.21.15.175:0/2030857809 shutdown_connections 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 --2- 172.21.15.175:0/2030857809 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f47300f6d10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 -- 172.21.15.175:0/2030857809 >> 172.21.15.175:0/2030857809 conn(0x7f47300f0120 msgr2=0x7f47300f2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 -- 172.21.15.175:0/2030857809 shutdown_connections 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.510+0000 7f4737340700 1 -- 172.21.15.175:0/2030857809 wait complete. 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f4737340700 1 Processor -- start 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f4737340700 1 -- start start 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f4737340700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f4730108690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f4737340700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730108b80 con 0x7f47300f68f0 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f473633e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f4730108690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f473633e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f4730108690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48084/0 (socket says 172.21.15.175:48084) 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f473633e700 1 -- 172.21.15.175:0/2397682540 learned_addr learned my addr 172.21.15.175:0/2397682540 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:37.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f473633e700 1 -- 172.21.15.175:0/2397682540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c005040 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.511+0000 7f473633e700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f4730108690 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f472c013510 tx=0x7f472c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.512+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f472c009320 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.512+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4730108d80 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.512+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f472c013f00 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.512+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f472c014dc0 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.512+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4730109210 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.513+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f472c01d3d0 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.513+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4730045ad0 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.513+0000 7f4726ffd700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f471c02ff60 0x7f471c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.513+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f472c041110 con 0x7f47300f68f0 2023-12-06T03:28:37.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.513+0000 7f4735b3d700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f471c02ff60 0x7f471c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:37.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.513+0000 7f4735b3d700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f471c02ff60 0x7f471c032420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f47200096a0 tx=0x7f4720006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:37.519 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.518+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f472c0095f0 con 0x7f47300f68f0 2023-12-06T03:28:37.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.723+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f473010a040 con 0x7f47300f68f0 2023-12-06T03:28:37.724 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.723+0000 7f4726ffd700 1 -- 172.21.15.175:0/2397682540 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f472c01cb10 con 0x7f47300f68f0 2023-12-06T03:28:37.725 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:37.725 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:37.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f471c02ff60 msgr2=0x7f471c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:37.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f471c02ff60 0x7f471c032420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f47200096a0 tx=0x7f4720006b40).stop 2023-12-06T03:28:37.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 msgr2=0x7f4730108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:37.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f4730108690 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f472c013510 tx=0x7f472c00b040).stop 2023-12-06T03:28:37.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 shutdown_connections 2023-12-06T03:28:37.726 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f471c02ff60 0x7f471c032420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:37.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 --2- 172.21.15.175:0/2397682540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47300f68f0 0x7f4730108690 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:37.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 >> 172.21.15.175:0/2397682540 conn(0x7f47300f0120 msgr2=0x7f47300f0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:37.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 shutdown_connections 2023-12-06T03:28:37.727 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:37.725+0000 7f4737340700 1 -- 172.21.15.175:0/2397682540 wait complete. 2023-12-06T03:28:37.727 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:37.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:36.472532+0000 mgr.smithi055.qjkkyb (mgr.14164) 223 : cluster [DBG] pgmap v211: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:38.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:38 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:37.724672+0000 mon.smithi055 (mon.0) 286 : audit [DBG] from='client.? 172.21.15.175:0/2397682540' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:39.141 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:39.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:38.472900+0000 mgr.smithi055.qjkkyb (mgr.14164) 224 : cluster [DBG] pgmap v212: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:40.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 -- 172.21.15.175:0/1038902997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 msgr2=0x7ff7d40f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:40.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 --2- 172.21.15.175:0/1038902997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d40f55c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7ff7c4009fb0 tx=0x7ff7c4009b30).stop 2023-12-06T03:28:40.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 -- 172.21.15.175:0/1038902997 shutdown_connections 2023-12-06T03:28:40.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 --2- 172.21.15.175:0/1038902997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d40f55c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:40.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 -- 172.21.15.175:0/1038902997 >> 172.21.15.175:0/1038902997 conn(0x7ff7d40f09a0 msgr2=0x7ff7d40f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:40.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 -- 172.21.15.175:0/1038902997 shutdown_connections 2023-12-06T03:28:40.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 -- 172.21.15.175:0/1038902997 wait complete. 2023-12-06T03:28:40.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.297+0000 7ff7d3fff700 1 Processor -- start 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d3fff700 1 -- start start 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d410b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4106fc0 con 0x7ff7d40f51a0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d410b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d410b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48104/0 (socket says 172.21.15.175:48104) 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d2ffd700 1 -- 172.21.15.175:0/2719487392 learned_addr learned my addr 172.21.15.175:0/2719487392 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d2ffd700 1 -- 172.21.15.175:0/2719487392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4005040 con 0x7ff7d40f51a0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d2ffd700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d410b5d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff7c4009830 tx=0x7ff7c400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7c4009320 con 0x7ff7d40f51a0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d4060220 con 0x7ff7d40f51a0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.298+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff7c40136c0 con 0x7ff7d40f51a0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.299+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7c401ee90 con 0x7ff7d40f51a0 2023-12-06T03:28:40.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.299+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d410bac0 con 0x7ff7d40f51a0 2023-12-06T03:28:40.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.299+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff7c401e420 con 0x7ff7d40f51a0 2023-12-06T03:28:40.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.299+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d4045ad0 con 0x7ff7d40f51a0 2023-12-06T03:28:40.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.299+0000 7ff7bb7fe700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff7bc02ff10 0x7ff7bc0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:40.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.300+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff7c4041010 con 0x7ff7d40f51a0 2023-12-06T03:28:40.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.300+0000 7ff7d27fc700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff7bc02ff10 0x7ff7bc0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:40.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.300+0000 7ff7d27fc700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff7bc02ff10 0x7ff7bc0323d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff7c80096a0 tx=0x7ff7c8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:40.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.305+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff7c40095f0 con 0x7ff7d40f51a0 2023-12-06T03:28:40.513 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.512+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff7d410c280 con 0x7ff7d40f51a0 2023-12-06T03:28:40.514 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7bb7fe700 1 -- 172.21.15.175:0/2719487392 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff7c401cb10 con 0x7ff7d40f51a0 2023-12-06T03:28:40.514 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:40.514 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:40.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff7bc02ff10 msgr2=0x7ff7bc0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff7bc02ff10 0x7ff7bc0323d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff7c80096a0 tx=0x7ff7c8006b40).stop 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 msgr2=0x7ff7d410b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d410b5d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7ff7c4009830 tx=0x7ff7c400b040).stop 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 shutdown_connections 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff7bc02ff10 0x7ff7bc0323d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 --2- 172.21.15.175:0/2719487392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40f51a0 0x7ff7d410b5d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 >> 172.21.15.175:0/2719487392 conn(0x7ff7d40f09a0 msgr2=0x7ff7d40f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 shutdown_connections 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:40.514+0000 7ff7d3fff700 1 -- 172.21.15.175:0/2719487392 wait complete. 2023-12-06T03:28:40.516 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:40.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:40 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:40.513901+0000 mon.smithi055 (mon.0) 287 : audit [DBG] from='client.? 172.21.15.175:0/2719487392' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:41.918 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:41.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:40.473237+0000 mgr.smithi055.qjkkyb (mgr.14164) 225 : cluster [DBG] pgmap v213: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 -- 172.21.15.175:0/192545273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 msgr2=0x7fb9e40f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 --2- 172.21.15.175:0/192545273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e40f6930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fb9d4009fb0 tx=0x7fb9d4009b30).stop 2023-12-06T03:28:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 -- 172.21.15.175:0/192545273 shutdown_connections 2023-12-06T03:28:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 --2- 172.21.15.175:0/192545273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e40f6930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 -- 172.21.15.175:0/192545273 >> 172.21.15.175:0/192545273 conn(0x7fb9e40f1c70 msgr2=0x7fb9e40f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 -- 172.21.15.175:0/192545273 shutdown_connections 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 -- 172.21.15.175:0/192545273 wait complete. 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 Processor -- start 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.083+0000 7fb9eb336700 1 -- start start 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9eb336700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e4106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9eb336700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e4106c20 con 0x7fb9e40f6510 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9e90d2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e4106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9e90d2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e4106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:48114/0 (socket says 172.21.15.175:48114) 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9e90d2700 1 -- 172.21.15.175:0/2398235068 learned_addr learned my addr 172.21.15.175:0/2398235068 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9e90d2700 1 -- 172.21.15.175:0/2398235068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4005040 con 0x7fb9e40f6510 2023-12-06T03:28:43.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9e90d2700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e4106730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fb9d4013510 tx=0x7fb9d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:43.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d4009320 con 0x7fb9e40f6510 2023-12-06T03:28:43.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9d4013f00 con 0x7fb9e40f6510 2023-12-06T03:28:43.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e4106e20 con 0x7fb9e40f6510 2023-12-06T03:28:43.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.084+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9d4014d90 con 0x7fb9e40f6510 2023-12-06T03:28:43.088 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.085+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e41072b0 con 0x7fb9e40f6510 2023-12-06T03:28:43.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.085+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fb9d40095f0 con 0x7fb9e40f6510 2023-12-06T03:28:43.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.085+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e4045ad0 con 0x7fb9e40f6510 2023-12-06T03:28:43.089 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.085+0000 7fb9d9ffb700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb9d002ff60 0x7fb9d0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:43.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.086+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fb9d4040fd0 con 0x7fb9e40f6510 2023-12-06T03:28:43.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.086+0000 7fb9e88d1700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb9d002ff60 0x7fb9d0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:43.090 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.086+0000 7fb9e88d1700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb9d002ff60 0x7fb9d0032420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fb9e00096a0 tx=0x7fb9e0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:43.091 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.091+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb9d4014420 con 0x7fb9e40f6510 2023-12-06T03:28:43.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.297+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb9e40fbf60 con 0x7fb9e40f6510 2023-12-06T03:28:43.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.297+0000 7fb9d9ffb700 1 -- 172.21.15.175:0/2398235068 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fb9d401d7f0 con 0x7fb9e40f6510 2023-12-06T03:28:43.299 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:43.299 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb9d002ff60 msgr2=0x7fb9d0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb9d002ff60 0x7fb9d0032420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fb9e00096a0 tx=0x7fb9e0006b40).stop 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 msgr2=0x7fb9e4106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e4106730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fb9d4013510 tx=0x7fb9d400b040).stop 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 shutdown_connections 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fb9d002ff60 0x7fb9d0032420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 --2- 172.21.15.175:0/2398235068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9e40f6510 0x7fb9e4106730 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 >> 172.21.15.175:0/2398235068 conn(0x7fb9e40f1c70 msgr2=0x7fb9e40f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 shutdown_connections 2023-12-06T03:28:43.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:43.299+0000 7fb9eb336700 1 -- 172.21.15.175:0/2398235068 wait complete. 2023-12-06T03:28:43.301 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:42.473549+0000 mgr.smithi055.qjkkyb (mgr.14164) 226 : cluster [DBG] pgmap v214: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:43.298321+0000 mon.smithi055 (mon.0) 288 : audit [DBG] from='client.? 172.21.15.175:0/2398235068' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:44.645 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:44.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:44.473889+0000 mgr.smithi055.qjkkyb (mgr.14164) 227 : cluster [DBG] pgmap v215: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:45.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.843+0000 7f0b0fcad700 1 -- 172.21.15.175:0/139726583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 msgr2=0x7f0b080f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:45.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.843+0000 7f0b0fcad700 1 --2- 172.21.15.175:0/139726583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b080f5830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f0afc009fb0 tx=0x7f0afc009b30).stop 2023-12-06T03:28:45.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.843+0000 7f0b0fcad700 1 -- 172.21.15.175:0/139726583 shutdown_connections 2023-12-06T03:28:45.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.843+0000 7f0b0fcad700 1 --2- 172.21.15.175:0/139726583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b080f5830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:45.845 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.843+0000 7f0b0fcad700 1 -- 172.21.15.175:0/139726583 >> 172.21.15.175:0/139726583 conn(0x7f0b080f0b70 msgr2=0x7f0b080f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0fcad700 1 -- 172.21.15.175:0/139726583 shutdown_connections 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0fcad700 1 -- 172.21.15.175:0/139726583 wait complete. 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0fcad700 1 Processor -- start 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0fcad700 1 -- start start 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0fcad700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b08109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0fcad700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08060220 con 0x7f0b080f5410 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0da49700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b08109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0da49700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b08109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55282/0 (socket says 172.21.15.175:55282) 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.844+0000 7f0b0da49700 1 -- 172.21.15.175:0/404696609 learned_addr learned my addr 172.21.15.175:0/404696609 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0b0da49700 1 -- 172.21.15.175:0/404696609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc005040 con 0x7f0b080f5410 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0b0da49700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b08109450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f0afc014000 tx=0x7f0afc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0afc0095f0 con 0x7f0b080f5410 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0afc007e70 con 0x7f0b080f5410 2023-12-06T03:28:45.846 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b08060420 con 0x7f0b080f5410 2023-12-06T03:28:45.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0afc014ea0 con 0x7f0b080f5410 2023-12-06T03:28:45.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.845+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b08109b00 con 0x7f0b080f5410 2023-12-06T03:28:45.847 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.846+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0afc025070 con 0x7f0b080f5410 2023-12-06T03:28:45.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.846+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b08059100 con 0x7f0b080f5410 2023-12-06T03:28:45.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.846+0000 7f0afa7fc700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0af402ff60 0x7f0af4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:45.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.846+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0afc0409f0 con 0x7f0b080f5410 2023-12-06T03:28:45.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.846+0000 7f0b0d248700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0af402ff60 0x7f0af4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:45.848 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.847+0000 7f0b0d248700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0af402ff60 0x7f0af4032420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f0b040096a0 tx=0x7f0b04006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:45.851 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:45.851+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0afc00f4e0 con 0x7f0b080f5410 2023-12-06T03:28:46.055 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.055+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0b08045ad0 con 0x7f0b080f5410 2023-12-06T03:28:46.056 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.055+0000 7f0afa7fc700 1 -- 172.21.15.175:0/404696609 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0afc00f300 con 0x7f0b080f5410 2023-12-06T03:28:46.056 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:46.056 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0af402ff60 msgr2=0x7f0af4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0af402ff60 0x7f0af4032420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f0b040096a0 tx=0x7f0b04006b40).stop 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 msgr2=0x7f0b08109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b08109450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f0afc014000 tx=0x7f0afc00b040).stop 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 shutdown_connections 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f0af402ff60 0x7f0af4032420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:46.057 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 --2- 172.21.15.175:0/404696609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b080f5410 0x7f0b08109450 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 >> 172.21.15.175:0/404696609 conn(0x7f0b080f0b70 msgr2=0x7f0b080f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 shutdown_connections 2023-12-06T03:28:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:46.057+0000 7f0b0fcad700 1 -- 172.21.15.175:0/404696609 wait complete. 2023-12-06T03:28:46.058 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:46.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:46 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:46.055565+0000 mon.smithi055 (mon.0) 289 : audit [DBG] from='client.? 172.21.15.175:0/404696609' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:47.443 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:47.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:47.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:46.474239+0000 mgr.smithi055.qjkkyb (mgr.14164) 228 : cluster [DBG] pgmap v216: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:48.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.619+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/1052756899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 msgr2=0x7f2b440f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:48.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.619+0000 7f2b4ab2a700 1 --2- 172.21.15.175:0/1052756899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b440f6930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f2b38009fb0 tx=0x7f2b38009b30).stop 2023-12-06T03:28:48.623 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.619+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/1052756899 shutdown_connections 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.619+0000 7f2b4ab2a700 1 --2- 172.21.15.175:0/1052756899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b440f6930 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.619+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/1052756899 >> 172.21.15.175:0/1052756899 conn(0x7f2b440f1c70 msgr2=0x7f2b440f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.620+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/1052756899 shutdown_connections 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.620+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/1052756899 wait complete. 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.620+0000 7f2b4ab2a700 1 Processor -- start 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.620+0000 7f2b4ab2a700 1 -- start start 2023-12-06T03:28:48.624 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.620+0000 7f2b4ab2a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b44106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:48.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.620+0000 7f2b4ab2a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b44106c20 con 0x7f2b440f6510 2023-12-06T03:28:48.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b488c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b44106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:48.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b488c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b44106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55308/0 (socket says 172.21.15.175:55308) 2023-12-06T03:28:48.625 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b488c6700 1 -- 172.21.15.175:0/2900667174 learned_addr learned my addr 172.21.15.175:0/2900667174 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b488c6700 1 -- 172.21.15.175:0/2900667174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38005040 con 0x7f2b440f6510 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b488c6700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b44106730 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f2b38013510 tx=0x7f2b3800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b38009320 con 0x7f2b440f6510 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b44106e20 con 0x7f2b440f6510 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b38013f00 con 0x7f2b440f6510 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b38014d90 con 0x7f2b440f6510 2023-12-06T03:28:48.626 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.621+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b441072b0 con 0x7f2b440f6510 2023-12-06T03:28:48.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.622+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2b380095f0 con 0x7f2b440f6510 2023-12-06T03:28:48.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.622+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b44045ad0 con 0x7f2b440f6510 2023-12-06T03:28:48.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.622+0000 7f2b417fa700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2b3402ff60 0x7f2b34032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:48.628 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.622+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2b38041000 con 0x7f2b440f6510 2023-12-06T03:28:48.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.623+0000 7f2b43fff700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2b3402ff60 0x7f2b34032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:48.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.623+0000 7f2b43fff700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2b3402ff60 0x7f2b34032420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f2b300096a0 tx=0x7f2b30006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:48.629 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.627+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2b38014420 con 0x7f2b440f6510 2023-12-06T03:28:48.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.834+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2b440fbf60 con 0x7f2b440f6510 2023-12-06T03:28:48.836 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.835+0000 7f2b417fa700 1 -- 172.21.15.175:0/2900667174 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2b3801d7f0 con 0x7f2b440f6510 2023-12-06T03:28:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:48.837 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:48.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2b3402ff60 msgr2=0x7f2b34032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:48.839 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2b3402ff60 0x7f2b34032420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f2b300096a0 tx=0x7f2b30006b40).stop 2023-12-06T03:28:48.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 msgr2=0x7f2b44106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:48.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b44106730 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f2b38013510 tx=0x7f2b3800b040).stop 2023-12-06T03:28:48.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 shutdown_connections 2023-12-06T03:28:48.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2b3402ff60 0x7f2b34032420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:48.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 --2- 172.21.15.175:0/2900667174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b440f6510 0x7f2b44106730 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:48.840 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 >> 172.21.15.175:0/2900667174 conn(0x7f2b440f1c70 msgr2=0x7f2b440f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:48.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 shutdown_connections 2023-12-06T03:28:48.841 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:48.837+0000 7f2b4ab2a700 1 -- 172.21.15.175:0/2900667174 wait complete. 2023-12-06T03:28:48.841 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:48.474623+0000 mgr.smithi055.qjkkyb (mgr.14164) 229 : cluster [DBG] pgmap v217: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:49.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:48.836171+0000 mon.smithi055 (mon.0) 290 : audit [DBG] from='client.? 172.21.15.175:0/2900667174' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:50.235 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:50.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 -- 172.21.15.175:0/1711499055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 msgr2=0x7f5acc0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 --2- 172.21.15.175:0/1711499055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc0f6930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f5abc009fb0 tx=0x7f5abc009b30).stop 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 -- 172.21.15.175:0/1711499055 shutdown_connections 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 --2- 172.21.15.175:0/1711499055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc0f6930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 -- 172.21.15.175:0/1711499055 >> 172.21.15.175:0/1711499055 conn(0x7f5acc0f1c70 msgr2=0x7f5acc0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 -- 172.21.15.175:0/1711499055 shutdown_connections 2023-12-06T03:28:51.375 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.373+0000 7f5ad4037700 1 -- 172.21.15.175:0/1711499055 wait complete. 2023-12-06T03:28:51.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad4037700 1 Processor -- start 2023-12-06T03:28:51.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad4037700 1 -- start start 2023-12-06T03:28:51.376 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad4037700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad4037700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc106c20 con 0x7f5acc0f6510 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad1dd3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad1dd3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55330/0 (socket says 172.21.15.175:55330) 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.374+0000 7f5ad1dd3700 1 -- 172.21.15.175:0/239966887 learned_addr learned my addr 172.21.15.175:0/239966887 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ad1dd3700 1 -- 172.21.15.175:0/239966887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc005040 con 0x7f5acc0f6510 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ad1dd3700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc106730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f5abc013510 tx=0x7f5abc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5abc009320 con 0x7f5acc0f6510 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc106e20 con 0x7f5acc0f6510 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5abc013f00 con 0x7f5acc0f6510 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5abc014d90 con 0x7f5acc0f6510 2023-12-06T03:28:51.377 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.375+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc1072b0 con 0x7f5acc0f6510 2023-12-06T03:28:51.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.376+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f5abc0095f0 con 0x7f5acc0f6510 2023-12-06T03:28:51.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.376+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc045ad0 con 0x7f5acc0f6510 2023-12-06T03:28:51.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.376+0000 7f5ac27fc700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ab802ff60 0x7f5ab8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:51.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.376+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5abc040fe0 con 0x7f5acc0f6510 2023-12-06T03:28:51.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.376+0000 7f5ad15d2700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ab802ff60 0x7f5ab8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:51.378 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.376+0000 7f5ad15d2700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ab802ff60 0x7f5ab8032420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5ac80096a0 tx=0x7f5ac8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:51.382 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.381+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5abc01d7f0 con 0x7f5acc0f6510 2023-12-06T03:28:51.592 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.591+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5acc0fbf60 con 0x7f5acc0f6510 2023-12-06T03:28:51.593 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.592+0000 7f5ac27fc700 1 -- 172.21.15.175:0/239966887 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5abc010230 con 0x7f5acc0f6510 2023-12-06T03:28:51.593 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:51.593 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ab802ff60 msgr2=0x7f5ab8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ab802ff60 0x7f5ab8032420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5ac80096a0 tx=0x7f5ac8006b40).stop 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 msgr2=0x7f5acc106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc106730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f5abc013510 tx=0x7f5abc00b040).stop 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 shutdown_connections 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f5ab802ff60 0x7f5ab8032420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 --2- 172.21.15.175:0/239966887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5acc0f6510 0x7f5acc106730 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 >> 172.21.15.175:0/239966887 conn(0x7f5acc0f1c70 msgr2=0x7f5acc0f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 shutdown_connections 2023-12-06T03:28:51.595 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:51.593+0000 7f5ad4037700 1 -- 172.21.15.175:0/239966887 wait complete. 2023-12-06T03:28:51.596 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:50.474924+0000 mgr.smithi055.qjkkyb (mgr.14164) 230 : cluster [DBG] pgmap v218: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:52.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:28:51.242231+0000 mgr.smithi055.qjkkyb (mgr.14164) 231 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Deploy daemon node-exporter.smithi055 ... 2023-12-06T03:28:52.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Verifying port 9100 ... 2023-12-06T03:28:52.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Non-zero exit code 1 from systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055 2023-12-06T03:28:52.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: systemctl: stderr Job for ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055.service failed because a timeout was exceeded. 2023-12-06T03:28:52.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: systemctl: stderr See "systemctl status ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055.service" and "journalctl -xe" for details. 2023-12-06T03:28:52.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Traceback (most recent call last): 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8029, in 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: main() 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8017, in main 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: r = ctx.func(ctx) 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 1760, in _default_image 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: return func(ctx) 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 4267, in command_deploy 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ports=daemon_ports) 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 2608, in deploy_daemon 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: c, osd_fsid=osd_fsid, ports=ports) 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 2806, in deploy_daemon_units 2023-12-06T03:28:52.952 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: call_throws(ctx, ['systemctl', 'start', unit_name]) 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 1464, in call_throws 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: raise RuntimeError('Failed command: %s' % ' '.join(command)) 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: RuntimeError: Failed command: systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Traceback (most recent call last): 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1144, in _remote_connection 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: yield (conn, connr) 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1059, in _run_cephadm 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: code, '\n'.join(err))) 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Deploy daemon node-exporter.smithi055 ... 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Verifying port 9100 ... 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Non-zero exit code 1 from systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: systemctl: stderr Job for ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055.service failed because a timeout was exceeded. 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: systemctl: stderr See "systemctl status ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055.service" and "journalctl -xe" for details. 2023-12-06T03:28:52.953 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: Traceback (most recent call last): 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8029, in 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: main() 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 8017, in main 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: r = ctx.func(ctx) 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 1760, in _default_image 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: return func(ctx) 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 4267, in command_deploy 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: ports=daemon_ports) 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 2608, in deploy_daemon 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: c, osd_fsid=osd_fsid, ports=ports) 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 2806, in deploy_daemon_units 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: call_throws(ctx, ['systemctl', 'start', unit_name]) 2023-12-06T03:28:52.954 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: File "/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/cephadm.30cb78bdbbafb384af862e1c2292b944f15942b586128e91262b43e91e11ae90", line 1464, in call_throws 2023-12-06T03:28:52.955 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: raise RuntimeError('Failed command: %s' % ' '.join(command)) 2023-12-06T03:28:52.955 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: RuntimeError: Failed command: systemctl start ceph-5555246c-93e6-11ee-95a2-87774f69a715@node-exporter.smithi055 2023-12-06T03:28:52.955 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cephadm 2023-12-06T03:28:51.248040+0000 mgr.smithi055.qjkkyb (mgr.14164) 232 : cephadm [INF] Deploying daemon prometheus.smithi055 on smithi055 2023-12-06T03:28:52.955 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:52 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:51.592994+0000 mon.smithi055 (mon.0) 291 : audit [DBG] from='client.? 172.21.15.175:0/239966887' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:52.984 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:52.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:52.475262+0000 mgr.smithi055.qjkkyb (mgr.14164) 233 : cluster [DBG] pgmap v219: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:54.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 -- 172.21.15.175:0/4154822777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 msgr2=0x7f84d00f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:54.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 --2- 172.21.15.175:0/4154822777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d00f6930 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f84cc009fb0 tx=0x7f84cc009b30).stop 2023-12-06T03:28:54.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 -- 172.21.15.175:0/4154822777 shutdown_connections 2023-12-06T03:28:54.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 --2- 172.21.15.175:0/4154822777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d00f6930 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 -- 172.21.15.175:0/4154822777 >> 172.21.15.175:0/4154822777 conn(0x7f84d00f1c70 msgr2=0x7f84d00f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 -- 172.21.15.175:0/4154822777 shutdown_connections 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.169+0000 7f84d7f34700 1 -- 172.21.15.175:0/4154822777 wait complete. 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d7f34700 1 Processor -- start 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d7f34700 1 -- start start 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d7f34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d0106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:54.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d7f34700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0106c20 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d5cd0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d0106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d5cd0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d0106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55340/0 (socket says 172.21.15.175:55340) 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.170+0000 7f84d5cd0700 1 -- 172.21.15.175:0/1646452414 learned_addr learned my addr 172.21.15.175:0/1646452414 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84d5cd0700 1 -- 172.21.15.175:0/1646452414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc005040 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84d5cd0700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d0106730 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f84cc013510 tx=0x7f84cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84cc009320 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d0106e20 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84cc013f00 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84cc014d90 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.171+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d01072b0 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.172+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f84cc0095f0 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.172+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d0045ad0 con 0x7f84d00f6510 2023-12-06T03:28:54.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.172+0000 7f84c67fc700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f84bc02ff60 0x7f84bc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:54.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.172+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f84cc0410c0 con 0x7f84d00f6510 2023-12-06T03:28:54.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.172+0000 7f84d54cf700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f84bc02ff60 0x7f84bc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:54.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.173+0000 7f84d54cf700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f84bc02ff60 0x7f84bc032420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f84c00096a0 tx=0x7f84c0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:54.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.177+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f84cc014420 con 0x7f84d00f6510 2023-12-06T03:28:54.385 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.384+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f84d00fbf60 con 0x7f84d00f6510 2023-12-06T03:28:54.386 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.385+0000 7f84c67fc700 1 -- 172.21.15.175:0/1646452414 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f84cc010230 con 0x7f84d00f6510 2023-12-06T03:28:54.386 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:54.387 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f84bc02ff60 msgr2=0x7f84bc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f84bc02ff60 0x7f84bc032420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f84c00096a0 tx=0x7f84c0006b40).stop 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 msgr2=0x7f84d0106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d0106730 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f84cc013510 tx=0x7f84cc00b040).stop 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 shutdown_connections 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f84bc02ff60 0x7f84bc032420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:54.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 --2- 172.21.15.175:0/1646452414 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84d00f6510 0x7f84d0106730 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:54.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 >> 172.21.15.175:0/1646452414 conn(0x7f84d00f1c70 msgr2=0x7f84d00f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:54.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 shutdown_connections 2023-12-06T03:28:54.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:54.387+0000 7f84d7f34700 1 -- 172.21.15.175:0/1646452414 wait complete. 2023-12-06T03:28:54.389 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:54.386244+0000 mon.smithi055 (mon.0) 292 : audit [DBG] from='client.? 172.21.15.175:0/1646452414' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:54.475501+0000 mgr.smithi055.qjkkyb (mgr.14164) 234 : cluster [DBG] pgmap v220: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:55.770 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:55.771 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 -- 172.21.15.175:0/3765635113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 msgr2=0x7ff0e00f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 --2- 172.21.15.175:0/3765635113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e00f55c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7ff0d4009fb0 tx=0x7ff0d4009b30).stop 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 -- 172.21.15.175:0/3765635113 shutdown_connections 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 --2- 172.21.15.175:0/3765635113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e00f55c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 -- 172.21.15.175:0/3765635113 >> 172.21.15.175:0/3765635113 conn(0x7ff0e00f09a0 msgr2=0x7ff0e00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 -- 172.21.15.175:0/3765635113 shutdown_connections 2023-12-06T03:28:56.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.972+0000 7ff0e4956700 1 -- 172.21.15.175:0/3765635113 wait complete. 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.973+0000 7ff0e4956700 1 Processor -- start 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.973+0000 7ff0e4956700 1 -- start start 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.973+0000 7ff0e4956700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e01064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.973+0000 7ff0e4956700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e01069a0 con 0x7ff0e00f51a0 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0ded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e01064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0ded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e01064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55094/0 (socket says 172.21.15.175:55094) 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0ded9d700 1 -- 172.21.15.175:0/2414206962 learned_addr learned my addr 172.21.15.175:0/2414206962 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0ded9d700 1 -- 172.21.15.175:0/2414206962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d4005040 con 0x7ff0e00f51a0 2023-12-06T03:28:56.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0ded9d700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e01064b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff0d4013510 tx=0x7ff0d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:56.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0d4009320 con 0x7ff0e00f51a0 2023-12-06T03:28:56.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e0106ba0 con 0x7ff0e00f51a0 2023-12-06T03:28:56.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0d4013f00 con 0x7ff0e00f51a0 2023-12-06T03:28:56.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0d4014dc0 con 0x7ff0e00f51a0 2023-12-06T03:28:56.977 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.974+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e0107030 con 0x7ff0e00f51a0 2023-12-06T03:28:56.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.975+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e0045ad0 con 0x7ff0e00f51a0 2023-12-06T03:28:56.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.975+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff0d401d3d0 con 0x7ff0e00f51a0 2023-12-06T03:28:56.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.975+0000 7ff0c77fe700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff0c802ff60 0x7ff0c8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:56.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.976+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7ff0d4041320 con 0x7ff0e00f51a0 2023-12-06T03:28:56.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.976+0000 7ff0de59c700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff0c802ff60 0x7ff0c8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:56.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.976+0000 7ff0de59c700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff0c802ff60 0x7ff0c8032420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7ff0d00096a0 tx=0x7ff0d0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:56.981 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:56.980+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff0d401cb10 con 0x7ff0e00f51a0 2023-12-06T03:28:57.187 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.185+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff0e0107e80 con 0x7ff0e00f51a0 2023-12-06T03:28:57.187 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.186+0000 7ff0c77fe700 1 -- 172.21.15.175:0/2414206962 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff0d401cb10 con 0x7ff0e00f51a0 2023-12-06T03:28:57.187 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:57.188 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff0c802ff60 msgr2=0x7ff0c8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff0c802ff60 0x7ff0c8032420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7ff0d00096a0 tx=0x7ff0d0006b40).stop 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 msgr2=0x7ff0e01064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e01064b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff0d4013510 tx=0x7ff0d400b040).stop 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 shutdown_connections 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7ff0c802ff60 0x7ff0c8032420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 --2- 172.21.15.175:0/2414206962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0e00f51a0 0x7ff0e01064b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 >> 172.21.15.175:0/2414206962 conn(0x7ff0e00f09a0 msgr2=0x7ff0e00f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:57.189 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 shutdown_connections 2023-12-06T03:28:57.190 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:57.188+0000 7ff0e4956700 1 -- 172.21.15.175:0/2414206962 wait complete. 2023-12-06T03:28:57.190 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:28:57.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:56.475800+0000 mgr.smithi055.qjkkyb (mgr.14164) 235 : cluster [DBG] pgmap v221: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:57.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:57.187160+0000 mon.smithi055 (mon.0) 293 : audit [DBG] from='client.? 172.21.15.175:0/2414206962' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:28:58.589 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:28:58.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:28:59.750 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/909286310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 msgr2=0x7f2f080f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:59.750 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 --2- 172.21.15.175:0/909286310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f080f66c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f2ef0009fb0 tx=0x7f2ef0009b30).stop 2023-12-06T03:28:59.750 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/909286310 shutdown_connections 2023-12-06T03:28:59.750 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 --2- 172.21.15.175:0/909286310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f080f66c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/909286310 >> 172.21.15.175:0/909286310 conn(0x7f2f080f1aa0 msgr2=0x7f2f080f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/909286310 shutdown_connections 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.747+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/909286310 wait complete. 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f0ce0e700 1 Processor -- start 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f0ce0e700 1 -- start start 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f0ce0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f081064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f0ce0e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081069a0 con 0x7f2f080f62a0 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f077fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f081064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f077fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f081064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55114/0 (socket says 172.21.15.175:55114) 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.748+0000 7f2f077fe700 1 -- 172.21.15.175:0/3032705776 learned_addr learned my addr 172.21.15.175:0/3032705776 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2f077fe700 1 -- 172.21.15.175:0/3032705776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef0005040 con 0x7f2f080f62a0 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2f077fe700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f081064b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f2ef0013510 tx=0x7f2ef000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ef0009320 con 0x7f2f080f62a0 2023-12-06T03:28:59.751 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f08106ba0 con 0x7f2f080f62a0 2023-12-06T03:28:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ef0013f00 con 0x7f2f080f62a0 2023-12-06T03:28:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ef0014dc0 con 0x7f2f080f62a0 2023-12-06T03:28:59.752 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.749+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f08107030 con 0x7f2f080f62a0 2023-12-06T03:28:59.754 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.750+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2ef00095f0 con 0x7f2f080f62a0 2023-12-06T03:28:59.754 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.750+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f08045ad0 con 0x7f2f080f62a0 2023-12-06T03:28:59.754 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.750+0000 7f2efffff700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2ef402ffb0 0x7f2ef4032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:28:59.755 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.750+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2ef0041050 con 0x7f2f080f62a0 2023-12-06T03:28:59.755 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.750+0000 7f2f06ffd700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2ef402ffb0 0x7f2ef4032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:28:59.755 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.751+0000 7f2f06ffd700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2ef402ffb0 0x7f2ef4032470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f2ef80096a0 tx=0x7f2ef8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:28:59.756 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.755+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2ef0014420 con 0x7f2f080f62a0 2023-12-06T03:28:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:28:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:28:58.476109+0000 mgr.smithi055.qjkkyb (mgr.14164) 236 : cluster [DBG] pgmap v222: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:28:59.961 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.960+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2f08107ee0 con 0x7f2f080f62a0 2023-12-06T03:28:59.962 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.961+0000 7f2efffff700 1 -- 172.21.15.175:0/3032705776 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f2ef0010230 con 0x7f2f080f62a0 2023-12-06T03:28:59.962 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:28:59.962 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2ef402ffb0 msgr2=0x7f2ef4032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2ef402ffb0 0x7f2ef4032470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f2ef80096a0 tx=0x7f2ef8006b40).stop 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 msgr2=0x7f2f081064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f081064b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f2ef0013510 tx=0x7f2ef000b040).stop 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 shutdown_connections 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f2ef402ffb0 0x7f2ef4032470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.962+0000 7f2f0ce0e700 1 --2- 172.21.15.175:0/3032705776 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f080f62a0 0x7f2f081064b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.963+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 >> 172.21.15.175:0/3032705776 conn(0x7f2f080f1aa0 msgr2=0x7f2f080f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.963+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 shutdown_connections 2023-12-06T03:28:59.964 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:28:59.963+0000 7f2f0ce0e700 1 -- 172.21.15.175:0/3032705776 wait complete. 2023-12-06T03:28:59.965 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:00.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:00 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:28:59.962223+0000 mon.smithi055 (mon.0) 294 : audit [DBG] from='client.? 172.21.15.175:0/3032705776' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:01.345 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:01.346 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:00.476380+0000 mgr.smithi055.qjkkyb (mgr.14164) 237 : cluster [DBG] pgmap v223: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:02.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 -- 172.21.15.175:0/3298812949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 msgr2=0x7f97680f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:02.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 --2- 172.21.15.175:0/3298812949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f97680f6930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f975c009fb0 tx=0x7f975c009b30).stop 2023-12-06T03:29:02.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 -- 172.21.15.175:0/3298812949 shutdown_connections 2023-12-06T03:29:02.579 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 --2- 172.21.15.175:0/3298812949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f97680f6930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:02.580 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 -- 172.21.15.175:0/3298812949 >> 172.21.15.175:0/3298812949 conn(0x7f97680f1c70 msgr2=0x7f97680f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:02.580 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 -- 172.21.15.175:0/3298812949 shutdown_connections 2023-12-06T03:29:02.580 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 -- 172.21.15.175:0/3298812949 wait complete. 2023-12-06T03:29:02.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.578+0000 7f976cdf5700 1 Processor -- start 2023-12-06T03:29:02.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976cdf5700 1 -- start start 2023-12-06T03:29:02.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976cdf5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f9768106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:02.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976cdf5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768106c20 con 0x7f97680f6510 2023-12-06T03:29:02.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f9768106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:02.581 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f9768106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55128/0 (socket says 172.21.15.175:55128) 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976659c700 1 -- 172.21.15.175:0/3970376035 learned_addr learned my addr 172.21.15.175:0/3970376035 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976659c700 1 -- 172.21.15.175:0/3970376035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c005040 con 0x7f97680f6510 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.579+0000 7f976659c700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f9768106730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f975c013510 tx=0x7f975c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f975c009320 con 0x7f97680f6510 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9768106e20 con 0x7f97680f6510 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f975c013f00 con 0x7f97680f6510 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f975c014d90 con 0x7f97680f6510 2023-12-06T03:29:02.582 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97681072b0 con 0x7f97680f6510 2023-12-06T03:29:02.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f975c0095f0 con 0x7f97680f6510 2023-12-06T03:29:02.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.580+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9768045ad0 con 0x7f97680f6510 2023-12-06T03:29:02.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.581+0000 7f9756ffd700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f975002ff60 0x7f9750032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:02.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.581+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f975c0410c0 con 0x7f97680f6510 2023-12-06T03:29:02.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.581+0000 7f9765d9b700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f975002ff60 0x7f9750032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:02.583 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.581+0000 7f9765d9b700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f975002ff60 0x7f9750032420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f97580096a0 tx=0x7f9758006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:02.587 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.586+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f975c014420 con 0x7f97680f6510 2023-12-06T03:29:02.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.789+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f97680fbf60 con 0x7f97680f6510 2023-12-06T03:29:02.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.789+0000 7f9756ffd700 1 -- 172.21.15.175:0/3970376035 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f975c01d7f0 con 0x7f97680f6510 2023-12-06T03:29:02.791 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:02.791 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:02.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f975002ff60 msgr2=0x7f9750032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:02.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f975002ff60 0x7f9750032420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f97580096a0 tx=0x7f9758006b40).stop 2023-12-06T03:29:02.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 msgr2=0x7f9768106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:02.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f9768106730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f975c013510 tx=0x7f975c00b040).stop 2023-12-06T03:29:02.792 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 shutdown_connections 2023-12-06T03:29:02.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f975002ff60 0x7f9750032420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:02.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 --2- 172.21.15.175:0/3970376035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97680f6510 0x7f9768106730 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:02.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 >> 172.21.15.175:0/3970376035 conn(0x7f97680f1c70 msgr2=0x7f97680f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:02.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 shutdown_connections 2023-12-06T03:29:02.793 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:02.791+0000 7f976cdf5700 1 -- 172.21.15.175:0/3970376035 wait complete. 2023-12-06T03:29:02.793 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:02.476739+0000 mgr.smithi055.qjkkyb (mgr.14164) 238 : cluster [DBG] pgmap v224: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:03.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:02.790655+0000 mon.smithi055 (mon.0) 295 : audit [DBG] from='client.? 172.21.15.175:0/3970376035' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:04.219 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:04.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:05.445 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 -- 172.21.15.175:0/3241557123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 msgr2=0x7fe7140f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 --2- 172.21.15.175:0/3241557123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7140f7d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fe704009fb0 tx=0x7fe704009b30).stop 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 -- 172.21.15.175:0/3241557123 shutdown_connections 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 --2- 172.21.15.175:0/3241557123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7140f7d00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 -- 172.21.15.175:0/3241557123 >> 172.21.15.175:0/3241557123 conn(0x7fe7140f0120 msgr2=0x7fe7140f2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 -- 172.21.15.175:0/3241557123 shutdown_connections 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 -- 172.21.15.175:0/3241557123 wait complete. 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.443+0000 7fe71359e700 1 Processor -- start 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71359e700 1 -- start start 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7141064e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7141069d0 con 0x7fe7140f78e0 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7141064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7141064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:36208/0 (socket says 172.21.15.175:36208) 2023-12-06T03:29:05.446 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71259c700 1 -- 172.21.15.175:0/1631111426 learned_addr learned my addr 172.21.15.175:0/1631111426 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:05.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71259c700 1 -- 172.21.15.175:0/1631111426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704005040 con 0x7fe7140f78e0 2023-12-06T03:29:05.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.444+0000 7fe71259c700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7141064e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fe704013510 tx=0x7fe70400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.445+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe704009320 con 0x7fe7140f78e0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.445+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe704013f00 con 0x7fe7140f78e0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.445+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe714106bd0 con 0x7fe7140f78e0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.445+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe704014dc0 con 0x7fe7140f78e0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.445+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe714107060 con 0x7fe7140f78e0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.446+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fe7040095f0 con 0x7fe7140f78e0 2023-12-06T03:29:05.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.446+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe714045ad0 con 0x7fe7140f78e0 2023-12-06T03:29:05.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.446+0000 7fe702ffd700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe6fc02ff60 0x7fe6fc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:05.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.446+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fe704041000 con 0x7fe7140f78e0 2023-12-06T03:29:05.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.446+0000 7fe711d9b700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe6fc02ff60 0x7fe6fc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:05.449 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.446+0000 7fe711d9b700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe6fc02ff60 0x7fe6fc032420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe7080096a0 tx=0x7fe708006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:05.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:04.477223+0000 mgr.smithi055.qjkkyb (mgr.14164) 239 : cluster [DBG] pgmap v225: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:05.452 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.451+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe70401c790 con 0x7fe7140f78e0 2023-12-06T03:29:05.657 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.656+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe714107f10 con 0x7fe7140f78e0 2023-12-06T03:29:05.658 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.657+0000 7fe702ffd700 1 -- 172.21.15.175:0/1631111426 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe704010230 con 0x7fe7140f78e0 2023-12-06T03:29:05.658 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:05.658 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe6fc02ff60 msgr2=0x7fe6fc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe6fc02ff60 0x7fe6fc032420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe7080096a0 tx=0x7fe708006b40).stop 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 msgr2=0x7fe7141064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7141064e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fe704013510 tx=0x7fe70400b040).stop 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 shutdown_connections 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe6fc02ff60 0x7fe6fc032420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 --2- 172.21.15.175:0/1631111426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7140f78e0 0x7fe7141064e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.658+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 >> 172.21.15.175:0/1631111426 conn(0x7fe7140f0120 msgr2=0x7fe7140f0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.659+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 shutdown_connections 2023-12-06T03:29:05.660 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:05.659+0000 7fe71359e700 1 -- 172.21.15.175:0/1631111426 wait complete. 2023-12-06T03:29:05.661 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:06.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:06 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:05.658087+0000 mon.smithi055 (mon.0) 296 : audit [DBG] from='client.? 172.21.15.175:0/1631111426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:07.025 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:07.026 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:07.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:06.477648+0000 mgr.smithi055.qjkkyb (mgr.14164) 240 : cluster [DBG] pgmap v226: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:08.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 -- 172.21.15.175:0/308620255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 msgr2=0x7f83380f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:08.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 --2- 172.21.15.175:0/308620255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83380f55c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f832c009fb0 tx=0x7f832c009b30).stop 2023-12-06T03:29:08.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 -- 172.21.15.175:0/308620255 shutdown_connections 2023-12-06T03:29:08.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 --2- 172.21.15.175:0/308620255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83380f55c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:08.171 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 -- 172.21.15.175:0/308620255 >> 172.21.15.175:0/308620255 conn(0x7f83380f09a0 msgr2=0x7f83380f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 -- 172.21.15.175:0/308620255 shutdown_connections 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 -- 172.21.15.175:0/308620255 wait complete. 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 Processor -- start 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.169+0000 7f833df45700 1 -- start start 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833df45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83381064a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833df45700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338106990 con 0x7f83380f51a0 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833cf43700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83381064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833cf43700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83381064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:36228/0 (socket says 172.21.15.175:36228) 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833cf43700 1 -- 172.21.15.175:0/3736224520 learned_addr learned my addr 172.21.15.175:0/3736224520 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:08.172 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833cf43700 1 -- 172.21.15.175:0/3736224520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c005040 con 0x7f83380f51a0 2023-12-06T03:29:08.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.170+0000 7f833cf43700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83381064a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f832c013510 tx=0x7f832c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:08.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.171+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f832c009320 con 0x7f83380f51a0 2023-12-06T03:29:08.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.171+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338106b90 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.171+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f832c013f00 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.171+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f832c014dc0 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.171+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8338107020 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.172+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f832c0095f0 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.172+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338045ad0 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.172+0000 7f83357fa700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f832802ff60 0x7f8328032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.172+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f832c041040 con 0x7f83380f51a0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.172+0000 7f8337fff700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f832802ff60 0x7f8328032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:08.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.172+0000 7f8337fff700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f832802ff60 0x7f8328032420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f83240096a0 tx=0x7f8324006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:08.178 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.177+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f832c014420 con 0x7f83380f51a0 2023-12-06T03:29:08.388 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.387+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8338107ed0 con 0x7f83380f51a0 2023-12-06T03:29:08.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.388+0000 7f83357fa700 1 -- 172.21.15.175:0/3736224520 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f832c010230 con 0x7f83380f51a0 2023-12-06T03:29:08.389 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:08.389 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:08.390 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f832802ff60 msgr2=0x7f8328032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f832802ff60 0x7f8328032420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f83240096a0 tx=0x7f8324006b40).stop 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 msgr2=0x7f83381064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83381064a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f832c013510 tx=0x7f832c00b040).stop 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 shutdown_connections 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f832802ff60 0x7f8328032420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 --2- 172.21.15.175:0/3736224520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83380f51a0 0x7f83381064a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 >> 172.21.15.175:0/3736224520 conn(0x7f83380f09a0 msgr2=0x7f833805fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 shutdown_connections 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:08.389+0000 7f833df45700 1 -- 172.21.15.175:0/3736224520 wait complete. 2023-12-06T03:29:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:08.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:08 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:08.388833+0000 mon.smithi055 (mon.0) 297 : audit [DBG] from='client.? 172.21.15.175:0/3736224520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:09.772 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:09.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:08.478029+0000 mgr.smithi055.qjkkyb (mgr.14164) 241 : cluster [DBG] pgmap v227: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:10.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 -- 172.21.15.175:0/2030539183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 msgr2=0x7f71600f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:10.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 --2- 172.21.15.175:0/2030539183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71600f66c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f7154009fb0 tx=0x7f7154009b30).stop 2023-12-06T03:29:10.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 -- 172.21.15.175:0/2030539183 shutdown_connections 2023-12-06T03:29:10.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 --2- 172.21.15.175:0/2030539183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71600f66c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:10.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 -- 172.21.15.175:0/2030539183 >> 172.21.15.175:0/2030539183 conn(0x7f71600f1aa0 msgr2=0x7f71600f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:10.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 -- 172.21.15.175:0/2030539183 shutdown_connections 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 -- 172.21.15.175:0/2030539183 wait complete. 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.982+0000 7f715f59e700 1 Processor -- start 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715f59e700 1 -- start start 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71601064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71601069a0 con 0x7f71600f62a0 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71601064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71601064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:36242/0 (socket says 172.21.15.175:36242) 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715e59c700 1 -- 172.21.15.175:0/668119278 learned_addr learned my addr 172.21.15.175:0/668119278 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715e59c700 1 -- 172.21.15.175:0/668119278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154005040 con 0x7f71600f62a0 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f715e59c700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71601064b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f7154013510 tx=0x7f715400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:10.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.983+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7154009320 con 0x7f71600f62a0 2023-12-06T03:29:10.987 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.984+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7154013f00 con 0x7f71600f62a0 2023-12-06T03:29:10.987 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.984+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7154014dc0 con 0x7f71600f62a0 2023-12-06T03:29:10.987 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.984+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7160106ba0 con 0x7f71600f62a0 2023-12-06T03:29:10.987 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.984+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7160107030 con 0x7f71600f62a0 2023-12-06T03:29:10.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.985+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f71540095f0 con 0x7f71600f62a0 2023-12-06T03:29:10.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.985+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7160045ad0 con 0x7f71600f62a0 2023-12-06T03:29:10.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.985+0000 7f714effd700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f714802ffb0 0x7f7148032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:10.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.985+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7154040ff0 con 0x7f71600f62a0 2023-12-06T03:29:10.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.985+0000 7f715dd9b700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f714802ffb0 0x7f7148032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:10.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.985+0000 7f715dd9b700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f714802ffb0 0x7f7148032470 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f71500096a0 tx=0x7f7150006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:10.991 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:10.990+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f715406d2b0 con 0x7f71600f62a0 2023-12-06T03:29:11.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.194+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7160107ee0 con 0x7f71600f62a0 2023-12-06T03:29:11.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.194+0000 7f714effd700 1 -- 172.21.15.175:0/668119278 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7154014420 con 0x7f71600f62a0 2023-12-06T03:29:11.195 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:11.196 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:11.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f714802ffb0 msgr2=0x7f7148032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:11.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f714802ffb0 0x7f7148032470 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f71500096a0 tx=0x7f7150006b40).stop 2023-12-06T03:29:11.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 msgr2=0x7f71601064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:11.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71601064b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f7154013510 tx=0x7f715400b040).stop 2023-12-06T03:29:11.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 shutdown_connections 2023-12-06T03:29:11.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f714802ffb0 0x7f7148032470 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:11.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 --2- 172.21.15.175:0/668119278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f71600f62a0 0x7f71601064b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:11.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 >> 172.21.15.175:0/668119278 conn(0x7f71600f1aa0 msgr2=0x7f71600f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:11.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 shutdown_connections 2023-12-06T03:29:11.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:11.196+0000 7f715f59e700 1 -- 172.21.15.175:0/668119278 wait complete. 2023-12-06T03:29:11.198 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:10.478321+0000 mgr.smithi055.qjkkyb (mgr.14164) 242 : cluster [DBG] pgmap v228: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:11.195454+0000 mon.smithi055 (mon.0) 298 : audit [DBG] from='client.? 172.21.15.175:0/668119278' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:12.602 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:12.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:13.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.785+0000 7f39fe5c6700 1 -- 172.21.15.175:0/3505136450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 msgr2=0x7f39f80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:13.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.785+0000 7f39fe5c6700 1 --2- 172.21.15.175:0/3505136450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f80f5830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f39e8009fb0 tx=0x7f39e8009b30).stop 2023-12-06T03:29:13.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 -- 172.21.15.175:0/3505136450 shutdown_connections 2023-12-06T03:29:13.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 --2- 172.21.15.175:0/3505136450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f80f5830 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:13.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 -- 172.21.15.175:0/3505136450 >> 172.21.15.175:0/3505136450 conn(0x7f39f80f0b70 msgr2=0x7f39f80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:13.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 -- 172.21.15.175:0/3505136450 shutdown_connections 2023-12-06T03:29:13.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 -- 172.21.15.175:0/3505136450 wait complete. 2023-12-06T03:29:13.788 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 Processor -- start 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 -- start start 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f8109420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.786+0000 7f39fe5c6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8060220 con 0x7f39f80f5410 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f8109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f8109420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:36260/0 (socket says 172.21.15.175:36260) 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f7fff700 1 -- 172.21.15.175:0/2900322101 learned_addr learned my addr 172.21.15.175:0/2900322101 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f7fff700 1 -- 172.21.15.175:0/2900322101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8005040 con 0x7f39f80f5410 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f7fff700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f8109420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f39e8013510 tx=0x7f39e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:13.789 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39e8009320 con 0x7f39f80f5410 2023-12-06T03:29:13.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8060420 con 0x7f39f80f5410 2023-12-06T03:29:13.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f39e8013f00 con 0x7f39f80f5410 2023-12-06T03:29:13.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39e8014d90 con 0x7f39f80f5410 2023-12-06T03:29:13.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.787+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f8109ad0 con 0x7f39f80f5410 2023-12-06T03:29:13.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.788+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f39e80095f0 con 0x7f39f80f5410 2023-12-06T03:29:13.790 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.788+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8059100 con 0x7f39f80f5410 2023-12-06T03:29:13.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.788+0000 7f39f4ff9700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f39e002ff60 0x7f39e0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:13.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.788+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f39e8040fc0 con 0x7f39f80f5410 2023-12-06T03:29:13.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.788+0000 7f39f77fe700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f39e002ff60 0x7f39e0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:13.791 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.789+0000 7f39f77fe700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f39e002ff60 0x7f39e0032420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f39ec0096a0 tx=0x7f39ec006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:13.794 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:13.793+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f39e80268c0 con 0x7f39f80f5410 2023-12-06T03:29:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:12.478601+0000 mgr.smithi055.qjkkyb (mgr.14164) 243 : cluster [DBG] pgmap v229: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:14.001 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.000+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f39f8045ad0 con 0x7f39f80f5410 2023-12-06T03:29:14.002 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.001+0000 7f39f4ff9700 1 -- 172.21.15.175:0/2900322101 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f39e8010230 con 0x7f39f80f5410 2023-12-06T03:29:14.002 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:14.002 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f39e002ff60 msgr2=0x7f39e0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f39e002ff60 0x7f39e0032420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f39ec0096a0 tx=0x7f39ec006b40).stop 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 msgr2=0x7f39f8109420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f8109420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f39e8013510 tx=0x7f39e800b040).stop 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 shutdown_connections 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f39e002ff60 0x7f39e0032420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 --2- 172.21.15.175:0/2900322101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39f80f5410 0x7f39f8109420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.002+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 >> 172.21.15.175:0/2900322101 conn(0x7f39f80f0b70 msgr2=0x7f39f80f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.003+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 shutdown_connections 2023-12-06T03:29:14.004 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:14.003+0000 7f39fe5c6700 1 -- 172.21.15.175:0/2900322101 wait complete. 2023-12-06T03:29:14.005 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:14.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:14 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:14.002020+0000 mon.smithi055 (mon.0) 299 : audit [DBG] from='client.? 172.21.15.175:0/2900322101' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:15.447 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:15.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:15.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:14.478899+0000 mgr.smithi055.qjkkyb (mgr.14164) 244 : cluster [DBG] pgmap v230: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.643+0000 7f904df07700 1 -- 172.21.15.175:0/1218327671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 msgr2=0x7f90480f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.643+0000 7f904df07700 1 --2- 172.21.15.175:0/1218327671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90480f90f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f9038009fb0 tx=0x7f9038009b30).stop 2023-12-06T03:29:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.643+0000 7f904df07700 1 -- 172.21.15.175:0/1218327671 shutdown_connections 2023-12-06T03:29:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.643+0000 7f904df07700 1 --2- 172.21.15.175:0/1218327671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90480f90f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:16.645 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.643+0000 7f904df07700 1 -- 172.21.15.175:0/1218327671 >> 172.21.15.175:0/1218327671 conn(0x7f90480f0b70 msgr2=0x7f90480f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:16.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.644+0000 7f904df07700 1 -- 172.21.15.175:0/1218327671 shutdown_connections 2023-12-06T03:29:16.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.644+0000 7f904df07700 1 -- 172.21.15.175:0/1218327671 wait complete. 2023-12-06T03:29:16.646 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.644+0000 7f904df07700 1 Processor -- start 2023-12-06T03:29:16.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.644+0000 7f904df07700 1 -- start start 2023-12-06T03:29:16.647 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f904df07700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90481066c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f904df07700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048106bb0 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f90477fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90481066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f90477fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90481066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55722/0 (socket says 172.21.15.175:55722) 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f90477fe700 1 -- 172.21.15.175:0/3402970333 learned_addr learned my addr 172.21.15.175:0/3402970333 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f90477fe700 1 -- 172.21.15.175:0/3402970333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038005040 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f90477fe700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90481066c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f9038013510 tx=0x7f903800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9038009320 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9038013f00 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9048106db0 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9038014dc0 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.645+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9048107240 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.646+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f90380095f0 con 0x7f9048060280 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.646+0000 7f902ffff700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f903002ff60 0x7f9030032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:16.648 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.646+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9048045ad0 con 0x7f9048060280 2023-12-06T03:29:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.646+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9038041000 con 0x7f9048060280 2023-12-06T03:29:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.647+0000 7f9046ffd700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f903002ff60 0x7f9030032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:16.649 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.647+0000 7f9046ffd700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f903002ff60 0x7f9030032420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f903c0096a0 tx=0x7f903c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:16.652 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.651+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f903806b2b0 con 0x7f9048060280 2023-12-06T03:29:16.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.856+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f90481080f0 con 0x7f9048060280 2023-12-06T03:29:16.858 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.857+0000 7f902ffff700 1 -- 172.21.15.175:0/3402970333 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f903802c090 con 0x7f9048060280 2023-12-06T03:29:16.859 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:16.859 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f903002ff60 msgr2=0x7f9030032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f903002ff60 0x7f9030032420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f903c0096a0 tx=0x7f903c006b40).stop 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 msgr2=0x7f90481066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90481066c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f9038013510 tx=0x7f903800b040).stop 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 shutdown_connections 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f903002ff60 0x7f9030032420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 --2- 172.21.15.175:0/3402970333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9048060280 0x7f90481066c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:16.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 >> 172.21.15.175:0/3402970333 conn(0x7f90480f0b70 msgr2=0x7f90480f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:16.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 shutdown_connections 2023-12-06T03:29:16.861 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:16.859+0000 7f904df07700 1 -- 172.21.15.175:0/3402970333 wait complete. 2023-12-06T03:29:16.861 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:16.479211+0000 mgr.smithi055.qjkkyb (mgr.14164) 245 : cluster [DBG] pgmap v231: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:17.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:16.858156+0000 mon.smithi055 (mon.0) 300 : audit [DBG] from='client.? 172.21.15.175:0/3402970333' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:18.257 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:18.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.391+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3893109858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 msgr2=0x7f12b40f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:19.394 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.391+0000 7f12ba8c0700 1 --2- 172.21.15.175:0/3893109858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b40f66c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f12a4009fb0 tx=0x7f12a4009b30).stop 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3893109858 shutdown_connections 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 --2- 172.21.15.175:0/3893109858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b40f66c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3893109858 >> 172.21.15.175:0/3893109858 conn(0x7f12b40f1ac0 msgr2=0x7f12b40f3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3893109858 shutdown_connections 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3893109858 wait complete. 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 Processor -- start 2023-12-06T03:29:19.395 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 -- start start 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b41064a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.392+0000 7f12ba8c0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4106990 con 0x7f12b40f62a0 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12b98be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b41064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12b98be700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b41064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55728/0 (socket says 172.21.15.175:55728) 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12b98be700 1 -- 172.21.15.175:0/3232102277 learned_addr learned my addr 172.21.15.175:0/3232102277 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12b98be700 1 -- 172.21.15.175:0/3232102277 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a4005040 con 0x7f12b40f62a0 2023-12-06T03:29:19.396 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12b98be700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b41064a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f12a4013510 tx=0x7f12a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:19.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12a4009320 con 0x7f12b40f62a0 2023-12-06T03:29:19.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b4106b90 con 0x7f12b40f62a0 2023-12-06T03:29:19.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12a4013f00 con 0x7f12b40f62a0 2023-12-06T03:29:19.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12a4014dc0 con 0x7f12b40f62a0 2023-12-06T03:29:19.397 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.393+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b4107020 con 0x7f12b40f62a0 2023-12-06T03:29:19.399 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.394+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f12a4014420 con 0x7f12b40f62a0 2023-12-06T03:29:19.399 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.394+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b4045ad0 con 0x7f12b40f62a0 2023-12-06T03:29:19.400 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.394+0000 7f12aa7fc700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12a002ff60 0x7f12a0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:19.400 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.394+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f12a4041050 con 0x7f12b40f62a0 2023-12-06T03:29:19.400 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.395+0000 7f12b90bd700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12a002ff60 0x7f12a0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:19.400 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.395+0000 7f12b90bd700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12a002ff60 0x7f12a0032420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f12b00096a0 tx=0x7f12b0006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:19.401 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.399+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f12a40095f0 con 0x7f12b40f62a0 2023-12-06T03:29:19.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.609+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f12b4107ed0 con 0x7f12b40f62a0 2023-12-06T03:29:19.611 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.609+0000 7f12aa7fc700 1 -- 172.21.15.175:0/3232102277 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f12a401cb10 con 0x7f12b40f62a0 2023-12-06T03:29:19.611 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:19.611 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12a002ff60 msgr2=0x7f12a0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12a002ff60 0x7f12a0032420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f12b00096a0 tx=0x7f12b0006b40).stop 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 msgr2=0x7f12b41064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b41064a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f12a4013510 tx=0x7f12a400b040).stop 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 shutdown_connections 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f12a002ff60 0x7f12a0032420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:19.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 --2- 172.21.15.175:0/3232102277 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12b40f62a0 0x7f12b41064a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:19.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 >> 172.21.15.175:0/3232102277 conn(0x7f12b40f1ac0 msgr2=0x7f12b40f2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:19.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 shutdown_connections 2023-12-06T03:29:19.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:19.611+0000 7f12ba8c0700 1 -- 172.21.15.175:0/3232102277 wait complete. 2023-12-06T03:29:19.613 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:19.897 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:18.479671+0000 mgr.smithi055.qjkkyb (mgr.14164) 246 : cluster [DBG] pgmap v232: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:20.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:20 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:19.610652+0000 mon.smithi055 (mon.0) 301 : audit [DBG] from='client.? 172.21.15.175:0/3232102277' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:21.011 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:21.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:20.480069+0000 mgr.smithi055.qjkkyb (mgr.14164) 247 : cluster [DBG] pgmap v233: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.234+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3400561978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 msgr2=0x7f85c00f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.234+0000 7f85c5ca2700 1 --2- 172.21.15.175:0/3400561978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c00f90f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f85b0009fb0 tx=0x7f85b0009b30).stop 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3400561978 shutdown_connections 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 --2- 172.21.15.175:0/3400561978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c00f90f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3400561978 >> 172.21.15.175:0/3400561978 conn(0x7f85c00f0b70 msgr2=0x7f85c00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3400561978 shutdown_connections 2023-12-06T03:29:22.237 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3400561978 wait complete. 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 Processor -- start 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.235+0000 7f85c5ca2700 1 -- start start 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85c5ca2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c0106690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85c5ca2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0106b80 con 0x7f85c0060280 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85bf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c0106690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85bf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c0106690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55744/0 (socket says 172.21.15.175:55744) 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85bf7fe700 1 -- 172.21.15.175:0/3740126752 learned_addr learned my addr 172.21.15.175:0/3740126752 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:22.238 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85bf7fe700 1 -- 172.21.15.175:0/3740126752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0005040 con 0x7f85c0060280 2023-12-06T03:29:22.239 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85bf7fe700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c0106690 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f85b0013510 tx=0x7f85b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:22.239 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b0009320 con 0x7f85c0060280 2023-12-06T03:29:22.239 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c0106d80 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85b0013f00 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.236+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85b0014dc0 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.237+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0107210 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.237+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f85b00095f0 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.237+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c0045ad0 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.237+0000 7f85a7fff700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f85a802ff60 0x7f85a8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.238+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f85b0041100 con 0x7f85c0060280 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.238+0000 7f85beffd700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f85a802ff60 0x7f85a8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:22.240 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.238+0000 7f85beffd700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f85a802ff60 0x7f85a8032420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f85b40096a0 tx=0x7f85b4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:22.244 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.242+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f85b0014420 con 0x7f85c0060280 2023-12-06T03:29:22.447 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.445+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f85c01080c0 con 0x7f85c0060280 2023-12-06T03:29:22.448 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.446+0000 7f85a7fff700 1 -- 172.21.15.175:0/3740126752 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f85b0010230 con 0x7f85c0060280 2023-12-06T03:29:22.450 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:22.450 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:22.452 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f85a802ff60 msgr2=0x7f85a8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:22.452 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f85a802ff60 0x7f85a8032420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f85b40096a0 tx=0x7f85b4006b40).stop 2023-12-06T03:29:22.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 msgr2=0x7f85c0106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c0106690 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f85b0013510 tx=0x7f85b000b040).stop 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 shutdown_connections 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f85a802ff60 0x7f85a8032420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 --2- 172.21.15.175:0/3740126752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85c0060280 0x7f85c0106690 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 >> 172.21.15.175:0/3740126752 conn(0x7f85c00f0b70 msgr2=0x7f85c00f1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 shutdown_connections 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:22.447+0000 7f85c5ca2700 1 -- 172.21.15.175:0/3740126752 wait complete. 2023-12-06T03:29:22.454 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:22.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:22.446868+0000 mon.smithi055 (mon.0) 302 : audit [DBG] from='client.? 172.21.15.175:0/3740126752' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:22.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:22 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:22.556883+0000 mon.smithi055 (mon.0) 303 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:29:23.843 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:23.843 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:22.480408+0000 mgr.smithi055.qjkkyb (mgr.14164) 248 : cluster [DBG] pgmap v234: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:23.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:22.559863+0000 mon.smithi055 (mon.0) 304 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 -- 172.21.15.175:0/3005395575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 msgr2=0x7f90f00f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 --2- 172.21.15.175:0/3005395575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f00f6930 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f90e4009fb0 tx=0x7f90e4009b30).stop 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 -- 172.21.15.175:0/3005395575 shutdown_connections 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 --2- 172.21.15.175:0/3005395575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f00f6930 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 -- 172.21.15.175:0/3005395575 >> 172.21.15.175:0/3005395575 conn(0x7f90f00f1c70 msgr2=0x7f90f00f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 -- 172.21.15.175:0/3005395575 shutdown_connections 2023-12-06T03:29:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.977+0000 7f90f7f25700 1 -- 172.21.15.175:0/3005395575 wait complete. 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f7f25700 1 Processor -- start 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f7f25700 1 -- start start 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f7f25700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f0106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f7f25700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f0106c20 con 0x7f90f00f6510 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f5cc1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f0106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f5cc1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f0106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:55768/0 (socket says 172.21.15.175:55768) 2023-12-06T03:29:24.984 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f5cc1700 1 -- 172.21.15.175:0/3086626178 learned_addr learned my addr 172.21.15.175:0/3086626178 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.978+0000 7f90f5cc1700 1 -- 172.21.15.175:0/3086626178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e4005040 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.979+0000 7f90f5cc1700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f0106730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f90e4013510 tx=0x7f90e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.979+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90e4009320 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.979+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90e4013f00 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.979+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f0106e20 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.979+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90e4014d90 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.979+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f01072b0 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.980+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f90e40263a0 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.980+0000 7f90e27fc700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f90dc02ff60 0x7f90dc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.980+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f0045ad0 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.980+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f90e40412f0 con 0x7f90f00f6510 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.980+0000 7f90f54c0700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f90dc02ff60 0x7f90dc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:24.985 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.980+0000 7f90f54c0700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f90dc02ff60 0x7f90dc032420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f90ec0096a0 tx=0x7f90ec006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:24.986 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:24.985+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f90e401cb50 con 0x7f90f00f6510 2023-12-06T03:29:25.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.192+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f90f00fc050 con 0x7f90f00f6510 2023-12-06T03:29:25.194 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.193+0000 7f90e27fc700 1 -- 172.21.15.175:0/3086626178 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f90e401cd60 con 0x7f90f00f6510 2023-12-06T03:29:25.194 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:25.194 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:25.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f90dc02ff60 msgr2=0x7f90dc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:25.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f90dc02ff60 0x7f90dc032420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f90ec0096a0 tx=0x7f90ec006b40).stop 2023-12-06T03:29:25.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 msgr2=0x7f90f0106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f0106730 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f90e4013510 tx=0x7f90e400b040).stop 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 shutdown_connections 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f90dc02ff60 0x7f90dc032420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 --2- 172.21.15.175:0/3086626178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00f6510 0x7f90f0106730 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 >> 172.21.15.175:0/3086626178 conn(0x7f90f00f1c70 msgr2=0x7f90f00f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 shutdown_connections 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:25.194+0000 7f90f7f25700 1 -- 172.21.15.175:0/3086626178 wait complete. 2023-12-06T03:29:25.196 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:25.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:25 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:24.480703+0000 mgr.smithi055.qjkkyb (mgr.14164) 249 : cluster [DBG] pgmap v235: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:26.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:26 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:25.193767+0000 mon.smithi055 (mon.0) 305 : audit [DBG] from='client.? 172.21.15.175:0/3086626178' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:26.582 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:26.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:27.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:27 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:26.481173+0000 mgr.smithi055.qjkkyb (mgr.14164) 250 : cluster [DBG] pgmap v236: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:27.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1715634615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 msgr2=0x7fd2180f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:27.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 --2- 172.21.15.175:0/1715634615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2180f66c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fd208009fb0 tx=0x7fd208009b30).stop 2023-12-06T03:29:27.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1715634615 shutdown_connections 2023-12-06T03:29:27.772 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 --2- 172.21.15.175:0/1715634615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2180f66c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:27.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1715634615 >> 172.21.15.175:0/1715634615 conn(0x7fd2180f1aa0 msgr2=0x7fd2180f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:27.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1715634615 shutdown_connections 2023-12-06T03:29:27.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1715634615 wait complete. 2023-12-06T03:29:27.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 Processor -- start 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.771+0000 7fd21cfcd700 1 -- start start 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd21cfcd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2181086b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd21cfcd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218108ba0 con 0x7fd2180f62a0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd2177fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2181086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd2177fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2181086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35284/0 (socket says 172.21.15.175:35284) 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd2177fe700 1 -- 172.21.15.175:0/1507419617 learned_addr learned my addr 172.21.15.175:0/1507419617 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd2177fe700 1 -- 172.21.15.175:0/1507419617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208005040 con 0x7fd2180f62a0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd2177fe700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2181086b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd208013510 tx=0x7fd20800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd208009320 con 0x7fd2180f62a0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd218108da0 con 0x7fd2180f62a0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd208013f00 con 0x7fd2180f62a0 2023-12-06T03:29:27.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.772+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd208014dc0 con 0x7fd2180f62a0 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.773+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd218109230 con 0x7fd2180f62a0 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.773+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd20801d3d0 con 0x7fd2180f62a0 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.773+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd218045ad0 con 0x7fd2180f62a0 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.773+0000 7fd1fffff700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd20002ff60 0x7fd200032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.774+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd208041100 con 0x7fd2180f62a0 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.774+0000 7fd216ffd700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd20002ff60 0x7fd200032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:27.776 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.774+0000 7fd216ffd700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd20002ff60 0x7fd200032420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd20c0096a0 tx=0x7fd20c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:27.779 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.779+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd2080095f0 con 0x7fd2180f62a0 2023-12-06T03:29:27.987 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.986+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd21810a0e0 con 0x7fd2180f62a0 2023-12-06T03:29:27.988 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.987+0000 7fd1fffff700 1 -- 172.21.15.175:0/1507419617 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd20801cb10 con 0x7fd2180f62a0 2023-12-06T03:29:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:27.988 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd20002ff60 msgr2=0x7fd200032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd20002ff60 0x7fd200032420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd20c0096a0 tx=0x7fd20c006b40).stop 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 msgr2=0x7fd2181086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2181086b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fd208013510 tx=0x7fd20800b040).stop 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 shutdown_connections 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd20002ff60 0x7fd200032420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 --2- 172.21.15.175:0/1507419617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2180f62a0 0x7fd2181086b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 >> 172.21.15.175:0/1507419617 conn(0x7fd2180f1aa0 msgr2=0x7fd2180f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 shutdown_connections 2023-12-06T03:29:27.990 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:27.988+0000 7fd21cfcd700 1 -- 172.21.15.175:0/1507419617 wait complete. 2023-12-06T03:29:27.991 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:28.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:28 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:27.987924+0000 mon.smithi055 (mon.0) 306 : audit [DBG] from='client.? 172.21.15.175:0/1507419617' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:29.374 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:29.375 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:29.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:29 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:28.481534+0000 mgr.smithi055.qjkkyb (mgr.14164) 251 : cluster [DBG] pgmap v237: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:30.605 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 -- 172.21.15.175:0/1796781591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 msgr2=0x7fe0d80f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 --2- 172.21.15.175:0/1796781591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d80f66c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7fe0c8009fb0 tx=0x7fe0c8009b30).stop 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 -- 172.21.15.175:0/1796781591 shutdown_connections 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 --2- 172.21.15.175:0/1796781591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d80f66c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 -- 172.21.15.175:0/1796781591 >> 172.21.15.175:0/1796781591 conn(0x7fe0d80f1aa0 msgr2=0x7fe0d80f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 -- 172.21.15.175:0/1796781591 shutdown_connections 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 -- 172.21.15.175:0/1796781591 wait complete. 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 Processor -- start 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.604+0000 7fe0dc952700 1 -- start start 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0dc952700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d81064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:30.606 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0dc952700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d81069a0 con 0x7fe0d80f62a0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d81064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d81064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35312/0 (socket says 172.21.15.175:35312) 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0d77fe700 1 -- 172.21.15.175:0/1390666566 learned_addr learned my addr 172.21.15.175:0/1390666566 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0d77fe700 1 -- 172.21.15.175:0/1390666566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c8005040 con 0x7fe0d80f62a0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0d77fe700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d81064b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fe0c8013510 tx=0x7fe0c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0c8009320 con 0x7fe0d80f62a0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d8106ba0 con 0x7fe0d80f62a0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0c8013f00 con 0x7fe0d80f62a0 2023-12-06T03:29:30.608 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0c8014dc0 con 0x7fe0d80f62a0 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.605+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d8107030 con 0x7fe0d80f62a0 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.606+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fe0c80095f0 con 0x7fe0d80f62a0 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.606+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d8045ad0 con 0x7fe0d80f62a0 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.606+0000 7fe0bffff700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe0c002ff60 0x7fe0c0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.606+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fe0c8041040 con 0x7fe0d80f62a0 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.607+0000 7fe0d6ffd700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe0c002ff60 0x7fe0c0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:30.609 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.607+0000 7fe0d6ffd700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe0c002ff60 0x7fe0c0032420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fe0cc0096a0 tx=0x7fe0cc006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:30.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.612+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe0c8014420 con 0x7fe0d80f62a0 2023-12-06T03:29:30.818 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.817+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe0d8107ee0 con 0x7fe0d80f62a0 2023-12-06T03:29:30.819 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.817+0000 7fe0bffff700 1 -- 172.21.15.175:0/1390666566 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fe0c8014420 con 0x7fe0d80f62a0 2023-12-06T03:29:30.819 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:30.819 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:30.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe0c002ff60 msgr2=0x7fe0c0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:30.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe0c002ff60 0x7fe0c0032420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fe0cc0096a0 tx=0x7fe0cc006b40).stop 2023-12-06T03:29:30.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 msgr2=0x7fe0d81064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:30.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d81064b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7fe0c8013510 tx=0x7fe0c800b040).stop 2023-12-06T03:29:30.820 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 shutdown_connections 2023-12-06T03:29:30.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fe0c002ff60 0x7fe0c0032420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:30.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 --2- 172.21.15.175:0/1390666566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d80f62a0 0x7fe0d81064b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:30.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 >> 172.21.15.175:0/1390666566 conn(0x7fe0d80f1aa0 msgr2=0x7fe0d80f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:30.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 shutdown_connections 2023-12-06T03:29:30.821 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:30.819+0000 7fe0dc952700 1 -- 172.21.15.175:0/1390666566 wait complete. 2023-12-06T03:29:30.821 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:30.481957+0000 mgr.smithi055.qjkkyb (mgr.14164) 252 : cluster [DBG] pgmap v238: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:31.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:31 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:30.818696+0000 mon.smithi055 (mon.0) 307 : audit [DBG] from='client.? 172.21.15.175:0/1390666566' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:32.182 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:32.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:33.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 -- 172.21.15.175:0/2187177549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 msgr2=0x7f8c800f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:33.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 --2- 172.21.15.175:0/2187177549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c800f5830 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f8c70009fb0 tx=0x7f8c70009b30).stop 2023-12-06T03:29:33.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 -- 172.21.15.175:0/2187177549 shutdown_connections 2023-12-06T03:29:33.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 --2- 172.21.15.175:0/2187177549 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c800f5830 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:33.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 -- 172.21.15.175:0/2187177549 >> 172.21.15.175:0/2187177549 conn(0x7f8c800f0b70 msgr2=0x7f8c800f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:33.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 -- 172.21.15.175:0/2187177549 shutdown_connections 2023-12-06T03:29:33.306 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.304+0000 7f8c8803b700 1 -- 172.21.15.175:0/2187177549 wait complete. 2023-12-06T03:29:33.308 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c8803b700 1 Processor -- start 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c8803b700 1 -- start start 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c8803b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c80106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c8803b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80106c20 con 0x7f8c800f5410 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c85dd7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c80106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c85dd7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c80106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:35320/0 (socket says 172.21.15.175:35320) 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c85dd7700 1 -- 172.21.15.175:0/592293688 learned_addr learned my addr 172.21.15.175:0/592293688 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c85dd7700 1 -- 172.21.15.175:0/592293688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70005040 con 0x7f8c800f5410 2023-12-06T03:29:33.309 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.305+0000 7f8c85dd7700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c80106730 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f8c70013510 tx=0x7f8c7000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.306+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c70009320 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.306+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c80106e20 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.306+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c70013f00 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.306+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c70014d90 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.306+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c801072b0 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.307+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8c700263a0 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.307+0000 7f8c767fc700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c6c02ff60 0x7f8c6c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.307+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8c700410b0 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.307+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80045ad0 con 0x7f8c800f5410 2023-12-06T03:29:33.310 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.307+0000 7f8c855d6700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c6c02ff60 0x7f8c6c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:33.311 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.308+0000 7f8c855d6700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c6c02ff60 0x7f8c6c032420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8c7c0096a0 tx=0x7f8c7c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:33.313 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.312+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8c70012920 con 0x7f8c800f5410 2023-12-06T03:29:33.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.522+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8c800faf50 con 0x7f8c800f5410 2023-12-06T03:29:33.524 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.522+0000 7f8c767fc700 1 -- 172.21.15.175:0/592293688 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8c70012330 con 0x7f8c800f5410 2023-12-06T03:29:33.524 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:33.524 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:33.525 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c6c02ff60 msgr2=0x7f8c6c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c6c02ff60 0x7f8c6c032420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f8c7c0096a0 tx=0x7f8c7c006b40).stop 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 msgr2=0x7f8c80106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c80106730 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f8c70013510 tx=0x7f8c7000b040).stop 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 shutdown_connections 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c6c02ff60 0x7f8c6c032420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 --2- 172.21.15.175:0/592293688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c800f5410 0x7f8c80106730 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 >> 172.21.15.175:0/592293688 conn(0x7f8c800f0b70 msgr2=0x7f8c800f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 shutdown_connections 2023-12-06T03:29:33.526 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:33.524+0000 7f8c8803b700 1 -- 172.21.15.175:0/592293688 wait complete. 2023-12-06T03:29:33.527 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:33.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:32.482430+0000 mgr.smithi055.qjkkyb (mgr.14164) 253 : cluster [DBG] pgmap v239: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:33.951 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:33 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:33.523747+0000 mon.smithi055 (mon.0) 308 : audit [DBG] from='client.? 172.21.15.175:0/592293688' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:34.885 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:34.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:35.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:35 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:34.482865+0000 mgr.smithi055.qjkkyb (mgr.14164) 254 : cluster [DBG] pgmap v240: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 -- 172.21.15.175:0/3180250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c0f47e0 msgr2=0x7f8f3c0f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 --2- 172.21.15.175:0/3180250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c0f47e0 0x7f8f3c0f4c00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f8f30009fb0 tx=0x7f8f30009b30).stop 2023-12-06T03:29:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 -- 172.21.15.175:0/3180250446 shutdown_connections 2023-12-06T03:29:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 --2- 172.21.15.175:0/3180250446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c0f47e0 0x7f8f3c0f4c00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:35.999 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 -- 172.21.15.175:0/3180250446 >> 172.21.15.175:0/3180250446 conn(0x7f8f3c0f0260 msgr2=0x7f8f3c0f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 -- 172.21.15.175:0/3180250446 shutdown_connections 2023-12-06T03:29:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.997+0000 7f8f42eab700 1 -- 172.21.15.175:0/3180250446 wait complete. 2023-12-06T03:29:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f42eab700 1 Processor -- start 2023-12-06T03:29:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f42eab700 1 -- start start 2023-12-06T03:29:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f42eab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 0x7f8f3c106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:36.000 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f42eab700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c107330 con 0x7f8f3c106a20 2023-12-06T03:29:36.002 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f40c47700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 0x7f8f3c106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:36.002 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f40c47700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 0x7f8f3c106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51886/0 (socket says 172.21.15.175:51886) 2023-12-06T03:29:36.002 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.998+0000 7f8f40c47700 1 -- 172.21.15.175:0/3530274481 learned_addr learned my addr 172.21.15.175:0/3530274481 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f40c47700 1 -- 172.21.15.175:0/3530274481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f30005040 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f40c47700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 0x7f8f3c106e40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f8f30009f80 tx=0x7f8f30009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f300136c0 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c107530 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f300095f0 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f30014420 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:35.999+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c108080 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.000+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8f30025060 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.000+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c059100 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.000+0000 7f8f397fa700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f2c02ff60 0x7f8f2c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.000+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8f30040d80 con 0x7f8f3c106a20 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.000+0000 7f8f3bfff700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f2c02ff60 0x7f8f2c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:36.003 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.001+0000 7f8f3bfff700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f2c02ff60 0x7f8f2c032420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8f280096a0 tx=0x7f8f28006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:36.006 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.005+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8f3001c950 con 0x7f8f3c106a20 2023-12-06T03:29:36.215 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.214+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8f3c0fc2f0 con 0x7f8f3c106a20 2023-12-06T03:29:36.216 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.214+0000 7f8f397fa700 1 -- 172.21.15.175:0/3530274481 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8f30042090 con 0x7f8f3c106a20 2023-12-06T03:29:36.216 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:36.216 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f2c02ff60 msgr2=0x7f8f2c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f2c02ff60 0x7f8f2c032420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f8f280096a0 tx=0x7f8f28006b40).stop 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 msgr2=0x7f8f3c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 0x7f8f3c106e40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f8f30009f80 tx=0x7f8f30009b30).stop 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 shutdown_connections 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8f2c02ff60 0x7f8f2c032420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 --2- 172.21.15.175:0/3530274481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c106a20 0x7f8f3c106e40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 >> 172.21.15.175:0/3530274481 conn(0x7f8f3c0f0260 msgr2=0x7f8f3c0f0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 shutdown_connections 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:36.216+0000 7f8f42eab700 1 -- 172.21.15.175:0/3530274481 wait complete. 2023-12-06T03:29:36.219 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:36.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:36 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:36.215591+0000 mon.smithi055 (mon.0) 309 : audit [DBG] from='client.? 172.21.15.175:0/3530274481' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:37.601 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:37.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:37.700 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:37 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:36.483244+0000 mgr.smithi055.qjkkyb (mgr.14164) 255 : cluster [DBG] pgmap v241: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:38.809 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 -- 172.21.15.175:0/2606377609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 msgr2=0x7f53380f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 --2- 172.21.15.175:0/2606377609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53380f66c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f532c009fb0 tx=0x7f532c009b30).stop 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 -- 172.21.15.175:0/2606377609 shutdown_connections 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 --2- 172.21.15.175:0/2606377609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53380f66c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 -- 172.21.15.175:0/2606377609 >> 172.21.15.175:0/2606377609 conn(0x7f53380f1aa0 msgr2=0x7f53380f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 -- 172.21.15.175:0/2606377609 shutdown_connections 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.807+0000 7f533d89f700 1 -- 172.21.15.175:0/2606377609 wait complete. 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.808+0000 7f533d89f700 1 Processor -- start 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.808+0000 7f533d89f700 1 -- start start 2023-12-06T03:29:38.810 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.808+0000 7f533d89f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53381064b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:38.811 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.808+0000 7f533d89f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53381069a0 con 0x7f53380f62a0 2023-12-06T03:29:38.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533c89d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53381064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:38.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533c89d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53381064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51910/0 (socket says 172.21.15.175:51910) 2023-12-06T03:29:38.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533c89d700 1 -- 172.21.15.175:0/1500461408 learned_addr learned my addr 172.21.15.175:0/1500461408 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:38.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533c89d700 1 -- 172.21.15.175:0/1500461408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c005040 con 0x7f53380f62a0 2023-12-06T03:29:38.812 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533c89d700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53381064b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f532c013510 tx=0x7f532c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f532c009320 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5338106ba0 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f532c013f00 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f532c014dc0 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.809+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338107030 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.810+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338045ad0 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.810+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f532c0095f0 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.810+0000 7f53357fa700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f532802ff60 0x7f5328032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.810+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f532c041000 con 0x7f53380f62a0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.811+0000 7f5337fff700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f532802ff60 0x7f5328032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:38.813 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.811+0000 7f5337fff700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f532802ff60 0x7f5328032420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f53240096a0 tx=0x7f5324006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:38.816 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:38.815+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f532c01c790 con 0x7f53380f62a0 2023-12-06T03:29:39.020 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.019+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5338107ee0 con 0x7f53380f62a0 2023-12-06T03:29:39.021 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.020+0000 7f53357fa700 1 -- 172.21.15.175:0/1500461408 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f532c010230 con 0x7f53380f62a0 2023-12-06T03:29:39.022 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:39.022 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:39.023 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.021+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f532802ff60 msgr2=0x7f5328032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.021+0000 7f533d89f700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f532802ff60 0x7f5328032420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f53240096a0 tx=0x7f5324006b40).stop 2023-12-06T03:29:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.021+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 msgr2=0x7f53381064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.021+0000 7f533d89f700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53381064b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f532c013510 tx=0x7f532c00b040).stop 2023-12-06T03:29:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.022+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 shutdown_connections 2023-12-06T03:29:39.024 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.022+0000 7f533d89f700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f532802ff60 0x7f5328032420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:39.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.022+0000 7f533d89f700 1 --2- 172.21.15.175:0/1500461408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53380f62a0 0x7f53381064b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:39.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.022+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 >> 172.21.15.175:0/1500461408 conn(0x7f53380f1aa0 msgr2=0x7f53380f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:39.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.022+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 shutdown_connections 2023-12-06T03:29:39.025 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:39.022+0000 7f533d89f700 1 -- 172.21.15.175:0/1500461408 wait complete. 2023-12-06T03:29:39.026 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:38.483549+0000 mgr.smithi055.qjkkyb (mgr.14164) 256 : cluster [DBG] pgmap v242: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:39.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:39 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:39.020940+0000 mon.smithi055 (mon.0) 310 : audit [DBG] from='client.? 172.21.15.175:0/1500461408' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:40.405 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:40.406 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:41.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 -- 172.21.15.175:0/2205170800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 msgr2=0x7f3b580f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:41.572 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 --2- 172.21.15.175:0/2205170800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b580f5830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f3b48009fb0 tx=0x7f3b48009b30).stop 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 -- 172.21.15.175:0/2205170800 shutdown_connections 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 --2- 172.21.15.175:0/2205170800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b580f5830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 -- 172.21.15.175:0/2205170800 >> 172.21.15.175:0/2205170800 conn(0x7f3b580f0b70 msgr2=0x7f3b580f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 -- 172.21.15.175:0/2205170800 shutdown_connections 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.568+0000 7f3b5e268700 1 -- 172.21.15.175:0/2205170800 wait complete. 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b5e268700 1 Processor -- start 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b5e268700 1 -- start start 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b5e268700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b58105630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b5e268700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58105b20 con 0x7f3b580f5410 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b58105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b58105630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51922/0 (socket says 172.21.15.175:51922) 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.569+0000 7f3b57fff700 1 -- 172.21.15.175:0/2743503824 learned_addr learned my addr 172.21.15.175:0/2743503824 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:41.573 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b57fff700 1 -- 172.21.15.175:0/2743503824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48005040 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b57fff700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b58105630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f3b48013510 tx=0x7f3b4800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b48009320 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b58105d20 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b48013f00 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b48014d90 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.570+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b581061b0 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.571+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f3b480095f0 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.571+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b58045ad0 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.571+0000 7f3b54ff9700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3b4002ffb0 0x7f3b40032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.571+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3b48041090 con 0x7f3b580f5410 2023-12-06T03:29:41.574 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.571+0000 7f3b577fe700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3b4002ffb0 0x7f3b40032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:41.575 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.572+0000 7f3b577fe700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3b4002ffb0 0x7f3b40032470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f3b4c0096a0 tx=0x7f3b4c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:41.577 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.576+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3b4806d2b0 con 0x7f3b580f5410 2023-12-06T03:29:41.783 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.782+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3b580fae60 con 0x7f3b580f5410 2023-12-06T03:29:41.784 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.783+0000 7f3b54ff9700 1 -- 172.21.15.175:0/2743503824 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3b48014420 con 0x7f3b580f5410 2023-12-06T03:29:41.784 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:41.784 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3b4002ffb0 msgr2=0x7f3b40032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3b4002ffb0 0x7f3b40032470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f3b4c0096a0 tx=0x7f3b4c006b40).stop 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 msgr2=0x7f3b58105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b58105630 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f3b48013510 tx=0x7f3b4800b040).stop 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 shutdown_connections 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f3b4002ffb0 0x7f3b40032470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 --2- 172.21.15.175:0/2743503824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b580f5410 0x7f3b58105630 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 >> 172.21.15.175:0/2743503824 conn(0x7f3b580f0b70 msgr2=0x7f3b580f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:41.786 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.784+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 shutdown_connections 2023-12-06T03:29:41.787 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:41.785+0000 7f3b5e268700 1 -- 172.21.15.175:0/2743503824 wait complete. 2023-12-06T03:29:41.787 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:41.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:41 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:40.483959+0000 mgr.smithi055.qjkkyb (mgr.14164) 257 : cluster [DBG] pgmap v243: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:42.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:42 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:41.783939+0000 mon.smithi055 (mon.0) 311 : audit [DBG] from='client.? 172.21.15.175:0/2743503824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:43.184 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:43.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:43.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:43 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:42.484306+0000 mgr.smithi055.qjkkyb (mgr.14164) 258 : cluster [DBG] pgmap v244: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:44.338 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.335+0000 7fab616c5700 1 -- 172.21.15.175:0/3413731280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 msgr2=0x7fab5c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:44.338 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.335+0000 7fab616c5700 1 --2- 172.21.15.175:0/3413731280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0f5830 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fab4c009fb0 tx=0x7fab4c009b30).stop 2023-12-06T03:29:44.338 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 -- 172.21.15.175:0/3413731280 shutdown_connections 2023-12-06T03:29:44.338 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 --2- 172.21.15.175:0/3413731280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0f5830 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:44.338 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 -- 172.21.15.175:0/3413731280 >> 172.21.15.175:0/3413731280 conn(0x7fab5c0f0b70 msgr2=0x7fab5c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:44.338 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 -- 172.21.15.175:0/3413731280 shutdown_connections 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 -- 172.21.15.175:0/3413731280 wait complete. 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 Processor -- start 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 -- start start 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.336+0000 7fab616c5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0fb820 con 0x7fab5c0f5410 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab5affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab5affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:51932/0 (socket says 172.21.15.175:51932) 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab5affd700 1 -- 172.21.15.175:0/2958337739 learned_addr learned my addr 172.21.15.175:0/2958337739 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab5affd700 1 -- 172.21.15.175:0/2958337739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c005040 con 0x7fab5c0f5410 2023-12-06T03:29:44.339 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab5affd700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0fd3a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fab4c013510 tx=0x7fab4c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:44.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab4c009320 con 0x7fab5c0f5410 2023-12-06T03:29:44.340 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0fb9c0 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab4c013f00 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab4c014d90 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.337+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c0fbe50 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.338+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fab4c0263a0 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.338+0000 7fab437fe700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fab4402ff60 0x7fab44032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.338+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fab4c041020 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.338+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c059100 con 0x7fab5c0f5410 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.338+0000 7fab5a7fc700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fab4402ff60 0x7fab44032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:44.341 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.339+0000 7fab5a7fc700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fab4402ff60 0x7fab44032420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fab500096a0 tx=0x7fab50006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:44.347 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.344+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fab4c012920 con 0x7fab5c0f5410 2023-12-06T03:29:44.556 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.555+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab5c045ad0 con 0x7fab5c0f5410 2023-12-06T03:29:44.556 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.555+0000 7fab437fe700 1 -- 172.21.15.175:0/2958337739 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fab4c02b020 con 0x7fab5c0f5410 2023-12-06T03:29:44.557 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:44.557 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:44.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fab4402ff60 msgr2=0x7fab44032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:44.558 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fab4402ff60 0x7fab44032420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fab500096a0 tx=0x7fab50006b40).stop 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 msgr2=0x7fab5c0fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0fd3a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fab4c013510 tx=0x7fab4c00b040).stop 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 shutdown_connections 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fab4402ff60 0x7fab44032420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 --2- 172.21.15.175:0/2958337739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab5c0f5410 0x7fab5c0fd3a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 >> 172.21.15.175:0/2958337739 conn(0x7fab5c0f0b70 msgr2=0x7fab5c05fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 shutdown_connections 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:44.557+0000 7fab616c5700 1 -- 172.21.15.175:0/2958337739 wait complete. 2023-12-06T03:29:44.559 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:45.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:44.484730+0000 mgr.smithi055.qjkkyb (mgr.14164) 259 : cluster [DBG] pgmap v245: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:45.451 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:45 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:44.556627+0000 mon.smithi055 (mon.0) 312 : audit [DBG] from='client.? 172.21.15.175:0/2958337739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:45.961 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:45.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 -- 172.21.15.175:0/638427230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 msgr2=0x7faf080f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 --2- 172.21.15.175:0/638427230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf080f6930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7faef0009fb0 tx=0x7faef0009b30).stop 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 -- 172.21.15.175:0/638427230 shutdown_connections 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 --2- 172.21.15.175:0/638427230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf080f6930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 -- 172.21.15.175:0/638427230 >> 172.21.15.175:0/638427230 conn(0x7faf080f1c70 msgr2=0x7faf080f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 -- 172.21.15.175:0/638427230 shutdown_connections 2023-12-06T03:29:47.139 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.137+0000 7faf0d4e6700 1 -- 172.21.15.175:0/638427230 wait complete. 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.138+0000 7faf0d4e6700 1 Processor -- start 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.138+0000 7faf0d4e6700 1 -- start start 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.138+0000 7faf0d4e6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf0806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.138+0000 7faf0d4e6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf08069e20 con 0x7faf080f6510 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf06ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf0806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf06ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf0806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:37188/0 (socket says 172.21.15.175:37188) 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf06ffd700 1 -- 172.21.15.175:0/1144598753 learned_addr learned my addr 172.21.15.175:0/1144598753 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf06ffd700 1 -- 172.21.15.175:0/1144598753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0005040 con 0x7faf080f6510 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf06ffd700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf0806b9a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7faef0013510 tx=0x7faef000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faef0009320 con 0x7faf080f6510 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf08069fc0 con 0x7faf080f6510 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faef0013f00 con 0x7faf080f6510 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faef0014d90 con 0x7faf080f6510 2023-12-06T03:29:47.141 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.139+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0806a450 con 0x7faf080f6510 2023-12-06T03:29:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.140+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7faef0014420 con 0x7faf080f6510 2023-12-06T03:29:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.140+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf08059100 con 0x7faf080f6510 2023-12-06T03:29:47.142 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.140+0000 7faeff7fe700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faef402ffb0 0x7faef4032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:47.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.140+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7faef00410c0 con 0x7faf080f6510 2023-12-06T03:29:47.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.140+0000 7faf067fc700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faef402ffb0 0x7faef4032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:47.143 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.141+0000 7faf067fc700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faef402ffb0 0x7faef4032470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7faef80096a0 tx=0x7faef8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:47.146 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.146+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7faef00095f0 con 0x7faf080f6510 2023-12-06T03:29:47.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.353+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faf080fbf60 con 0x7faf080f6510 2023-12-06T03:29:47.354 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.353+0000 7faeff7fe700 1 -- 172.21.15.175:0/1144598753 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7faef001db70 con 0x7faf080f6510 2023-12-06T03:29:47.355 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:47.355 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faef402ffb0 msgr2=0x7faef4032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faef402ffb0 0x7faef4032470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7faef80096a0 tx=0x7faef8006b40).stop 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 msgr2=0x7faf0806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf0806b9a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7faef0013510 tx=0x7faef000b040).stop 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 shutdown_connections 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faef402ffb0 0x7faef4032470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 --2- 172.21.15.175:0/1144598753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf080f6510 0x7faf0806b9a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 >> 172.21.15.175:0/1144598753 conn(0x7faf080f1c70 msgr2=0x7faf080f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 shutdown_connections 2023-12-06T03:29:47.356 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:47.355+0000 7faf0d4e6700 1 -- 172.21.15.175:0/1144598753 wait complete. 2023-12-06T03:29:47.357 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:46.485113+0000 mgr.smithi055.qjkkyb (mgr.14164) 260 : cluster [DBG] pgmap v246: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:47.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:47 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:47.354468+0000 mon.smithi055 (mon.0) 313 : audit [DBG] from='client.? 172.21.15.175:0/1144598753' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:48.729 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:48.730 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- 172.21.15.175:0/1739153229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 msgr2=0x7f4a180f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 --2- 172.21.15.175:0/1739153229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a180f66c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4a0c009fb0 tx=0x7f4a0c009b30).stop 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- 172.21.15.175:0/1739153229 shutdown_connections 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 --2- 172.21.15.175:0/1739153229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a180f66c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- 172.21.15.175:0/1739153229 >> 172.21.15.175:0/1739153229 conn(0x7f4a180f1aa0 msgr2=0x7f4a180f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- 172.21.15.175:0/1739153229 shutdown_connections 2023-12-06T03:29:49.882 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- 172.21.15.175:0/1739153229 wait complete. 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 Processor -- start 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- start start 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a1806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.881+0000 7f4a1759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18069e20 con 0x7f4a180f62a0 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a1806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a1806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:37214/0 (socket says 172.21.15.175:37214) 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1659c700 1 -- 172.21.15.175:0/358738850 learned_addr learned my addr 172.21.15.175:0/358738850 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:49.883 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1659c700 1 -- 172.21.15.175:0/358738850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c005040 con 0x7f4a180f62a0 2023-12-06T03:29:49.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1659c700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a1806b9a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f4a0c009830 tx=0x7f4a0c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:49.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a0c009320 con 0x7f4a180f62a0 2023-12-06T03:29:49.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a18069fc0 con 0x7f4a180f62a0 2023-12-06T03:29:49.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a0c0136c0 con 0x7f4a180f62a0 2023-12-06T03:29:49.884 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a0c01ee90 con 0x7f4a180f62a0 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.882+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1806a450 con 0x7f4a180f62a0 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.883+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f4a0c014440 con 0x7f4a180f62a0 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.883+0000 7f4a06ffd700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4a0002ff10 0x7f4a000323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.883+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4a0c041330 con 0x7f4a180f62a0 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.883+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18059100 con 0x7f4a180f62a0 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.884+0000 7f4a15d9b700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4a0002ff10 0x7f4a000323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:49.885 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.884+0000 7f4a15d9b700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4a0002ff10 0x7f4a000323d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f4a080096a0 tx=0x7f4a08006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:49.889 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:49.889+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4a0c0273f0 con 0x7f4a180f62a0 2023-12-06T03:29:49.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:49 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:48.485463+0000 mgr.smithi055.qjkkyb (mgr.14164) 261 : cluster [DBG] pgmap v247: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:50.093 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.093+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4a180fbce0 con 0x7f4a180f62a0 2023-12-06T03:29:50.094 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.094+0000 7f4a06ffd700 1 -- 172.21.15.175:0/358738850 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4a0c01c320 con 0x7f4a180f62a0 2023-12-06T03:29:50.095 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:50.095 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:50.096 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4a0002ff10 msgr2=0x7f4a000323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:50.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4a0002ff10 0x7f4a000323d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f4a080096a0 tx=0x7f4a08006b40).stop 2023-12-06T03:29:50.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 msgr2=0x7f4a1806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:50.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a1806b9a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f4a0c009830 tx=0x7f4a0c00b040).stop 2023-12-06T03:29:50.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 shutdown_connections 2023-12-06T03:29:50.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4a0002ff10 0x7f4a000323d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:50.097 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 --2- 172.21.15.175:0/358738850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a180f62a0 0x7f4a1806b9a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:50.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 >> 172.21.15.175:0/358738850 conn(0x7f4a180f1aa0 msgr2=0x7f4a180f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:50.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 shutdown_connections 2023-12-06T03:29:50.098 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:50.095+0000 7f4a1759e700 1 -- 172.21.15.175:0/358738850 wait complete. 2023-12-06T03:29:50.098 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:50.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:50 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:50.093678+0000 mon.smithi055 (mon.0) 314 : audit [DBG] from='client.? 172.21.15.175:0/358738850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:51.497 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:51.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:51.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:51 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:50.485886+0000 mgr.smithi055.qjkkyb (mgr.14164) 262 : cluster [DBG] pgmap v248: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:52.637 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 -- 172.21.15.175:0/2669902427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 msgr2=0x7f523c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:52.637 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 --2- 172.21.15.175:0/2669902427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c0f5830 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f5238009fb0 tx=0x7f5238009b30).stop 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 -- 172.21.15.175:0/2669902427 shutdown_connections 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 --2- 172.21.15.175:0/2669902427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c0f5830 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 -- 172.21.15.175:0/2669902427 >> 172.21.15.175:0/2669902427 conn(0x7f523c0f0b70 msgr2=0x7f523c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 -- 172.21.15.175:0/2669902427 shutdown_connections 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.634+0000 7f5244331700 1 -- 172.21.15.175:0/2669902427 wait complete. 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.635+0000 7f5244331700 1 Processor -- start 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.635+0000 7f5244331700 1 -- start start 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.635+0000 7f5244331700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c105630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.635+0000 7f5244331700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c105b20 con 0x7f523c0f5410 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f52420cd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:52.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f52420cd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c105630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:37240/0 (socket says 172.21.15.175:37240) 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f52420cd700 1 -- 172.21.15.175:0/3201574464 learned_addr learned my addr 172.21.15.175:0/3201574464 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f52420cd700 1 -- 172.21.15.175:0/3201574464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238005040 con 0x7f523c0f5410 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f52420cd700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c105630 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f5238013510 tx=0x7f523800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5238009320 con 0x7f523c0f5410 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523c105d20 con 0x7f523c0f5410 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5238013f00 con 0x7f523c0f5410 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5238014d90 con 0x7f523c0f5410 2023-12-06T03:29:52.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.636+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c1061b0 con 0x7f523c0f5410 2023-12-06T03:29:52.642 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.637+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f5238014420 con 0x7f523c0f5410 2023-12-06T03:29:52.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.637+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c045ad0 con 0x7f523c0f5410 2023-12-06T03:29:52.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.637+0000 7f5232ffd700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f522802ff60 0x7f5228032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:52.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.637+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5238040fb0 con 0x7f523c0f5410 2023-12-06T03:29:52.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.637+0000 7f52418cc700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f522802ff60 0x7f5228032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:52.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.638+0000 7f52418cc700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f522802ff60 0x7f5228032420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f522c0096a0 tx=0x7f522c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:52.643 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.642+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f52380095f0 con 0x7f523c0f5410 2023-12-06T03:29:52.851 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.850+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f523c0fae60 con 0x7f523c0f5410 2023-12-06T03:29:52.852 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.851+0000 7f5232ffd700 1 -- 172.21.15.175:0/3201574464 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f523801db70 con 0x7f523c0f5410 2023-12-06T03:29:52.852 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:52.852 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f522802ff60 msgr2=0x7f5228032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f522802ff60 0x7f5228032420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f522c0096a0 tx=0x7f522c006b40).stop 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 msgr2=0x7f523c105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c105630 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f5238013510 tx=0x7f523800b040).stop 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 shutdown_connections 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f522802ff60 0x7f5228032420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 --2- 172.21.15.175:0/3201574464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f523c0f5410 0x7f523c105630 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.852+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 >> 172.21.15.175:0/3201574464 conn(0x7f523c0f0b70 msgr2=0x7f523c0f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.853+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 shutdown_connections 2023-12-06T03:29:52.854 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:52.853+0000 7f5244331700 1 -- 172.21.15.175:0/3201574464 wait complete. 2023-12-06T03:29:52.855 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:52.486332+0000 mgr.smithi055.qjkkyb (mgr.14164) 263 : cluster [DBG] pgmap v249: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:53.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:53 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:52.852103+0000 mon.smithi055 (mon.0) 315 : audit [DBG] from='client.? 172.21.15.175:0/3201574464' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:54.237 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:54.237 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:55.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:55 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:54.486727+0000 mgr.smithi055.qjkkyb (mgr.14164) 264 : cluster [DBG] pgmap v250: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 -- 172.21.15.175:0/1870778727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 msgr2=0x7fd8500f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 --2- 172.21.15.175:0/1870778727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8500f9150 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fd840009fb0 tx=0x7fd840009b30).stop 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 -- 172.21.15.175:0/1870778727 shutdown_connections 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 --2- 172.21.15.175:0/1870778727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8500f9150 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 -- 172.21.15.175:0/1870778727 >> 172.21.15.175:0/1870778727 conn(0x7fd8500f0b70 msgr2=0x7fd8500f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 -- 172.21.15.175:0/1870778727 shutdown_connections 2023-12-06T03:29:55.453 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.451+0000 7fd85864e700 1 -- 172.21.15.175:0/1870778727 wait complete. 2023-12-06T03:29:55.454 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd85864e700 1 Processor -- start 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd85864e700 1 -- start start 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd85864e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8501066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd85864e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850106bd0 con 0x7fd850060220 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd8563ea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8501066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd8563ea700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8501066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41826/0 (socket says 172.21.15.175:41826) 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.452+0000 7fd8563ea700 1 -- 172.21.15.175:0/2179360437 learned_addr learned my addr 172.21.15.175:0/2179360437 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd8563ea700 1 -- 172.21.15.175:0/2179360437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840005040 con 0x7fd850060220 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd8563ea700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8501066e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fd840013510 tx=0x7fd84000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd840009320 con 0x7fd850060220 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850106dd0 con 0x7fd850060220 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd840013f00 con 0x7fd850060220 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd840014dc0 con 0x7fd850060220 2023-12-06T03:29:55.455 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.453+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd850107260 con 0x7fd850060220 2023-12-06T03:29:55.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.454+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fd8400095f0 con 0x7fd850060220 2023-12-06T03:29:55.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.454+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd850045ad0 con 0x7fd850060220 2023-12-06T03:29:55.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.454+0000 7fd846ffd700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd83c02ff60 0x7fd83c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:55.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.454+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd840041080 con 0x7fd850060220 2023-12-06T03:29:55.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.454+0000 7fd855be9700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd83c02ff60 0x7fd83c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:55.457 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.455+0000 7fd855be9700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd83c02ff60 0x7fd83c032420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd84c0096a0 tx=0x7fd84c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:55.460 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.459+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd84001c790 con 0x7fd850060220 2023-12-06T03:29:55.663 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.662+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd850108110 con 0x7fd850060220 2023-12-06T03:29:55.664 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.662+0000 7fd846ffd700 1 -- 172.21.15.175:0/2179360437 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fd840010230 con 0x7fd850060220 2023-12-06T03:29:55.664 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:55.664 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:55.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd83c02ff60 msgr2=0x7fd83c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:55.665 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd83c02ff60 0x7fd83c032420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd84c0096a0 tx=0x7fd84c006b40).stop 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 msgr2=0x7fd8501066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8501066e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fd840013510 tx=0x7fd84000b040).stop 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 shutdown_connections 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fd83c02ff60 0x7fd83c032420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 --2- 172.21.15.175:0/2179360437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd850060220 0x7fd8501066e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 >> 172.21.15.175:0/2179360437 conn(0x7fd8500f0b70 msgr2=0x7fd8500f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 shutdown_connections 2023-12-06T03:29:55.666 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:55.664+0000 7fd85864e700 1 -- 172.21.15.175:0/2179360437 wait complete. 2023-12-06T03:29:55.667 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:56.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:56 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:55.663662+0000 mon.smithi055 (mon.0) 316 : audit [DBG] from='client.? 172.21.15.175:0/2179360437' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:57.049 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:57.049 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:57.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:57 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:56.487056+0000 mgr.smithi055.qjkkyb (mgr.14164) 265 : cluster [DBG] pgmap v251: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 -- 172.21.15.175:0/4038711940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 msgr2=0x7f9fd80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 --2- 172.21.15.175:0/4038711940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd80f5830 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f9fcc009fb0 tx=0x7f9fcc009b30).stop 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 -- 172.21.15.175:0/4038711940 shutdown_connections 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 --2- 172.21.15.175:0/4038711940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd80f5830 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 -- 172.21.15.175:0/4038711940 >> 172.21.15.175:0/4038711940 conn(0x7f9fd80f0b70 msgr2=0x7f9fd80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 -- 172.21.15.175:0/4038711940 shutdown_connections 2023-12-06T03:29:58.193 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 -- 172.21.15.175:0/4038711940 wait complete. 2023-12-06T03:29:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.191+0000 7f9fdcb24700 1 Processor -- start 2023-12-06T03:29:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fdcb24700 1 -- start start 2023-12-06T03:29:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fdcb24700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd8105630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fdcb24700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd8105b20 con 0x7f9fd80f5410 2023-12-06T03:29:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fd659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd8105630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:58.195 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fd659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd8105630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41840/0 (socket says 172.21.15.175:41840) 2023-12-06T03:29:58.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fd659c700 1 -- 172.21.15.175:0/2883099601 learned_addr learned my addr 172.21.15.175:0/2883099601 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:29:58.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fd659c700 1 -- 172.21.15.175:0/2883099601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc005040 con 0x7f9fd80f5410 2023-12-06T03:29:58.197 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.192+0000 7f9fd659c700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd8105630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f9fcc013510 tx=0x7f9fcc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.193+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9fcc009320 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.193+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9fcc013f00 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.193+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd8105d20 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.193+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9fcc014d90 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.193+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd81061b0 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.193+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f9fcc0263a0 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.194+0000 7f9fc6ffd700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9fc002ffb0 0x7f9fc0032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.194+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd8045ad0 con 0x7f9fd80f5410 2023-12-06T03:29:58.198 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.194+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9fcc041290 con 0x7f9fd80f5410 2023-12-06T03:29:58.200 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.194+0000 7f9fd5d9b700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9fc002ffb0 0x7f9fc0032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:29:58.200 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.194+0000 7f9fd5d9b700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9fc002ffb0 0x7f9fc0032470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9fc80096a0 tx=0x7f9fc8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:29:58.200 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.199+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9fcc0126f0 con 0x7f9fd80f5410 2023-12-06T03:29:58.408 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.406+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9fd80faf50 con 0x7f9fd80f5410 2023-12-06T03:29:58.410 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.407+0000 7f9fc6ffd700 1 -- 172.21.15.175:0/2883099601 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f9fcc012330 con 0x7f9fd80f5410 2023-12-06T03:29:58.410 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:29:58.411 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:29:58.411 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9fc002ffb0 msgr2=0x7f9fc0032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:58.411 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9fc002ffb0 0x7f9fc0032470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9fc80096a0 tx=0x7f9fc8006b40).stop 2023-12-06T03:29:58.411 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 msgr2=0x7f9fd8105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:29:58.411 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd8105630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f9fcc013510 tx=0x7f9fcc00b040).stop 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 shutdown_connections 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f9fc002ffb0 0x7f9fc0032470 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 --2- 172.21.15.175:0/2883099601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd80f5410 0x7f9fd8105630 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 >> 172.21.15.175:0/2883099601 conn(0x7f9fd80f0b70 msgr2=0x7f9fd80f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 shutdown_connections 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:29:58.409+0000 7f9fdcb24700 1 -- 172.21.15.175:0/2883099601 wait complete. 2023-12-06T03:29:58.412 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:29:58.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:58 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:29:58.407906+0000 mon.smithi055 (mon.0) 317 : audit [DBG] from='client.? 172.21.15.175:0/2883099601' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:29:59.800 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:29:59.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:29:59.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:29:59 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:29:58.487400+0000 mgr.smithi055.qjkkyb (mgr.14164) 266 : cluster [DBG] pgmap v252: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:00.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 -- 172.21.15.175:0/1983456001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 msgr2=0x7f38980f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:00.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 --2- 172.21.15.175:0/1983456001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f38980f5830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f388c009fb0 tx=0x7f388c009b30).stop 2023-12-06T03:30:00.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 -- 172.21.15.175:0/1983456001 shutdown_connections 2023-12-06T03:30:00.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 --2- 172.21.15.175:0/1983456001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f38980f5830 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:00.972 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 -- 172.21.15.175:0/1983456001 >> 172.21.15.175:0/1983456001 conn(0x7f38980f0b70 msgr2=0x7f38980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 -- 172.21.15.175:0/1983456001 shutdown_connections 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.969+0000 7f389ec98700 1 -- 172.21.15.175:0/1983456001 wait complete. 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ec98700 1 Processor -- start 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ec98700 1 -- start start 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ec98700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f3898106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ec98700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3898106c20 con 0x7f38980f5410 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ca34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f3898106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ca34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f3898106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41860/0 (socket says 172.21.15.175:41860) 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.970+0000 7f389ca34700 1 -- 172.21.15.175:0/347130466 learned_addr learned my addr 172.21.15.175:0/347130466 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f389ca34700 1 -- 172.21.15.175:0/347130466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c005040 con 0x7f38980f5410 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f389ca34700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f3898106730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f388c013510 tx=0x7f388c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f388c009320 con 0x7f38980f5410 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f388c013f00 con 0x7f38980f5410 2023-12-06T03:30:00.973 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3898106e20 con 0x7f38980f5410 2023-12-06T03:30:00.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f388c014d90 con 0x7f38980f5410 2023-12-06T03:30:00.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.971+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38981072b0 con 0x7f38980f5410 2023-12-06T03:30:00.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.972+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f388c0263a0 con 0x7f38980f5410 2023-12-06T03:30:00.974 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.972+0000 7f38957fa700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f388802ff60 0x7f3888032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:00.975 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.972+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3898045ad0 con 0x7f38980f5410 2023-12-06T03:30:00.975 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.972+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f388c041380 con 0x7f38980f5410 2023-12-06T03:30:00.975 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.972+0000 7f3897fff700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f388802ff60 0x7f3888032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:00.976 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.973+0000 7f3897fff700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f388802ff60 0x7f3888032420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f38840096a0 tx=0x7f3884006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:00.978 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:00.977+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f388c01cca0 con 0x7f38980f5410 2023-12-06T03:30:01.183 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.182+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f38980fc080 con 0x7f38980f5410 2023-12-06T03:30:01.184 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.183+0000 7f38957fa700 1 -- 172.21.15.175:0/347130466 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f388c012330 con 0x7f38980f5410 2023-12-06T03:30:01.184 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:01.184 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f388802ff60 msgr2=0x7f3888032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f388802ff60 0x7f3888032420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f38840096a0 tx=0x7f3884006b40).stop 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 msgr2=0x7f3898106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f3898106730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f388c013510 tx=0x7f388c00b040).stop 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 shutdown_connections 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f388802ff60 0x7f3888032420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 --2- 172.21.15.175:0/347130466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38980f5410 0x7f3898106730 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 >> 172.21.15.175:0/347130466 conn(0x7f38980f0b70 msgr2=0x7f38980f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 shutdown_connections 2023-12-06T03:30:01.187 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:01.184+0000 7f389ec98700 1 -- 172.21.15.175:0/347130466 wait complete. 2023-12-06T03:30:01.187 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:00.487728+0000 mgr.smithi055.qjkkyb (mgr.14164) 267 : cluster [DBG] pgmap v253: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:01.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:01 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:01.184111+0000 mon.smithi055 (mon.0) 318 : audit [DBG] from='client.? 172.21.15.175:0/347130466' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:02.582 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:02.582 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- 172.21.15.175:0/866043459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 msgr2=0x7f4ba80f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 --2- 172.21.15.175:0/866043459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba80f55c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f4b98009fb0 tx=0x7f4b98009b30).stop 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- 172.21.15.175:0/866043459 shutdown_connections 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 --2- 172.21.15.175:0/866043459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba80f55c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- 172.21.15.175:0/866043459 >> 172.21.15.175:0/866043459 conn(0x7f4ba80f09a0 msgr2=0x7f4ba80f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- 172.21.15.175:0/866043459 shutdown_connections 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- 172.21.15.175:0/866043459 wait complete. 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 Processor -- start 2023-12-06T03:30:03.797 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- start start 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba810b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.794+0000 7f4baecf6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8106fc0 con 0x7f4ba80f51a0 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4badcf4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba810b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4badcf4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba810b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:41876/0 (socket says 172.21.15.175:41876) 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4badcf4700 1 -- 172.21.15.175:0/139102760 learned_addr learned my addr 172.21.15.175:0/139102760 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4badcf4700 1 -- 172.21.15.175:0/139102760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b98005040 con 0x7f4ba80f51a0 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4badcf4700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba810b5d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4b98009830 tx=0x7f4b9800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b98009320 con 0x7f4ba80f51a0 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba8060220 con 0x7f4ba80f51a0 2023-12-06T03:30:03.798 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b980136c0 con 0x7f4ba80f51a0 2023-12-06T03:30:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b9801ee90 con 0x7f4ba80f51a0 2023-12-06T03:30:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.795+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba810bac0 con 0x7f4ba80f51a0 2023-12-06T03:30:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.796+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba8045ad0 con 0x7f4ba80f51a0 2023-12-06T03:30:03.799 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.796+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f4b9801e420 con 0x7f4ba80f51a0 2023-12-06T03:30:03.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.796+0000 7f4b9e7fc700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b9402ff10 0x7f4b940323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:03.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.796+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f4b980410b0 con 0x7f4ba80f51a0 2023-12-06T03:30:03.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.797+0000 7f4bad4f3700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b9402ff10 0x7f4b940323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:03.800 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.797+0000 7f4bad4f3700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b9402ff10 0x7f4b940323d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4ba40096a0 tx=0x7f4ba4006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:03.802 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:03.801+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4b980095f0 con 0x7f4ba80f51a0 2023-12-06T03:30:03.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:03 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:02.488305+0000 mgr.smithi055.qjkkyb (mgr.14164) 268 : cluster [DBG] pgmap v254: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:04.007 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.006+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ba810c2b0 con 0x7f4ba80f51a0 2023-12-06T03:30:04.008 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.006+0000 7f4b9e7fc700 1 -- 172.21.15.175:0/139102760 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f4b9802c2d0 con 0x7f4ba80f51a0 2023-12-06T03:30:04.008 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:04.008 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b9402ff10 msgr2=0x7f4b940323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b9402ff10 0x7f4b940323d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4ba40096a0 tx=0x7f4ba4006b40).stop 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 msgr2=0x7f4ba810b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba810b5d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f4b98009830 tx=0x7f4b9800b040).stop 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 shutdown_connections 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f4b9402ff10 0x7f4b940323d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 --2- 172.21.15.175:0/139102760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ba80f51a0 0x7f4ba810b5d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:04.011 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 >> 172.21.15.175:0/139102760 conn(0x7f4ba80f09a0 msgr2=0x7f4ba80f1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:04.011 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 shutdown_connections 2023-12-06T03:30:04.011 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:04.008+0000 7f4baecf6700 1 -- 172.21.15.175:0/139102760 wait complete. 2023-12-06T03:30:04.011 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:04.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:04 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:04.007677+0000 mon.smithi055 (mon.0) 319 : audit [DBG] from='client.? 172.21.15.175:0/139102760' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:05.426 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:05.426 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:05.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:05 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:04.488680+0000 mgr.smithi055.qjkkyb (mgr.14164) 269 : cluster [DBG] pgmap v255: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:06.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 -- 172.21.15.175:0/1000478952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 msgr2=0x7f59a00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:06.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 --2- 172.21.15.175:0/1000478952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a00f5830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f5994009fb0 tx=0x7f5994009b30).stop 2023-12-06T03:30:06.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 -- 172.21.15.175:0/1000478952 shutdown_connections 2023-12-06T03:30:06.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 --2- 172.21.15.175:0/1000478952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a00f5830 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:06.559 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 -- 172.21.15.175:0/1000478952 >> 172.21.15.175:0/1000478952 conn(0x7f59a00f0b70 msgr2=0x7f59a00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 -- 172.21.15.175:0/1000478952 shutdown_connections 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.556+0000 7f59a6068700 1 -- 172.21.15.175:0/1000478952 wait complete. 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f59a6068700 1 Processor -- start 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f59a6068700 1 -- start start 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f59a6068700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a0109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f59a6068700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a0060240 con 0x7f59a00f5410 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f599f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a0109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f599f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a0109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58048/0 (socket says 172.21.15.175:58048) 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.557+0000 7f599f7fe700 1 -- 172.21.15.175:0/2404545736 learned_addr learned my addr 172.21.15.175:0/2404545736 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.558+0000 7f599f7fe700 1 -- 172.21.15.175:0/2404545736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5994005040 con 0x7f59a00f5410 2023-12-06T03:30:06.560 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.558+0000 7f599f7fe700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a0109430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f5994013510 tx=0x7f599400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:06.561 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5994009320 con 0x7f59a00f5410 2023-12-06T03:30:06.561 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a0060440 con 0x7f59a00f5410 2023-12-06T03:30:06.561 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a0109ae0 con 0x7f59a00f5410 2023-12-06T03:30:06.561 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5994013f00 con 0x7f59a00f5410 2023-12-06T03:30:06.561 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5994014d60 con 0x7f59a00f5410 2023-12-06T03:30:06.562 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f5994014420 con 0x7f59a00f5410 2023-12-06T03:30:06.562 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.559+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5980004fa0 con 0x7f59a00f5410 2023-12-06T03:30:06.562 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.560+0000 7f5987fff700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f598802ff60 0x7f5988032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:06.562 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.560+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f5994041240 con 0x7f59a00f5410 2023-12-06T03:30:06.562 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.560+0000 7f599effd700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f598802ff60 0x7f5988032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:06.562 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.560+0000 7f599effd700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f598802ff60 0x7f5988032420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f59900096a0 tx=0x7f5990006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:06.566 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.565+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5994012b20 con 0x7f59a00f5410 2023-12-06T03:30:06.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.770+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5980005670 con 0x7f59a00f5410 2023-12-06T03:30:06.773 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.771+0000 7f5987fff700 1 -- 172.21.15.175:0/2404545736 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f5994012b20 con 0x7f59a00f5410 2023-12-06T03:30:06.773 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:06.773 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:06.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.772+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f598802ff60 msgr2=0x7f5988032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:06.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.772+0000 7f59a6068700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f598802ff60 0x7f5988032420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f59900096a0 tx=0x7f5990006b40).stop 2023-12-06T03:30:06.774 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.772+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 msgr2=0x7f59a0109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a0109430 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f5994013510 tx=0x7f599400b040).stop 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 shutdown_connections 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f598802ff60 0x7f5988032420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 --2- 172.21.15.175:0/2404545736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59a00f5410 0x7f59a0109430 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 >> 172.21.15.175:0/2404545736 conn(0x7f59a00f0b70 msgr2=0x7f59a00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 shutdown_connections 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:06.773+0000 7f59a6068700 1 -- 172.21.15.175:0/2404545736 wait complete. 2023-12-06T03:30:06.775 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:06.488949+0000 mgr.smithi055.qjkkyb (mgr.14164) 270 : cluster [DBG] pgmap v256: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:07.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:07 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:06.772216+0000 mon.smithi055 (mon.0) 320 : audit [DBG] from='client.? 172.21.15.175:0/2404545736' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:08.153 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:08.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:09.298 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 -- 172.21.15.175:0/2663217107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 msgr2=0x7f70200f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 --2- 172.21.15.175:0/2663217107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f70200f5830 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f7010009fb0 tx=0x7f7010009b30).stop 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 -- 172.21.15.175:0/2663217107 shutdown_connections 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 --2- 172.21.15.175:0/2663217107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f70200f5830 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 -- 172.21.15.175:0/2663217107 >> 172.21.15.175:0/2663217107 conn(0x7f70200f0b70 msgr2=0x7f70200f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 -- 172.21.15.175:0/2663217107 shutdown_connections 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.296+0000 7f702851c700 1 -- 172.21.15.175:0/2663217107 wait complete. 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f702851c700 1 Processor -- start 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f702851c700 1 -- start start 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f702851c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f7020109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f702851c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7020060220 con 0x7f70200f5410 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f70262b8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f7020109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f70262b8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f7020109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58066/0 (socket says 172.21.15.175:58066) 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f70262b8700 1 -- 172.21.15.175:0/1356317977 learned_addr learned my addr 172.21.15.175:0/1356317977 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:09.299 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.297+0000 7f70262b8700 1 -- 172.21.15.175:0/1356317977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010005040 con 0x7f70200f5410 2023-12-06T03:30:09.300 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.298+0000 7f70262b8700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f7020109450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f7010014000 tx=0x7f701000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.298+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70100095f0 con 0x7f70200f5410 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.298+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7010007e70 con 0x7f70200f5410 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.298+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7020060420 con 0x7f70200f5410 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.298+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7010014ea0 con 0x7f70200f5410 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.298+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7020109b00 con 0x7f70200f5410 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.299+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7010025070 con 0x7f70200f5410 2023-12-06T03:30:09.301 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.299+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7020059100 con 0x7f70200f5410 2023-12-06T03:30:09.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.299+0000 7f7016ffd700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f700c02ff60 0x7f700c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:09.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.299+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7010040970 con 0x7f70200f5410 2023-12-06T03:30:09.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.299+0000 7f7025ab7700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f700c02ff60 0x7f700c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:09.302 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.299+0000 7f7025ab7700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f700c02ff60 0x7f700c032420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f701c0096a0 tx=0x7f701c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:09.305 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.304+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f70100079c0 con 0x7f70200f5410 2023-12-06T03:30:09.514 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.513+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7020045ad0 con 0x7f70200f5410 2023-12-06T03:30:09.515 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.514+0000 7f7016ffd700 1 -- 172.21.15.175:0/1356317977 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f701000f300 con 0x7f70200f5410 2023-12-06T03:30:09.515 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:09.516 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f700c02ff60 msgr2=0x7f700c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f700c02ff60 0x7f700c032420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f701c0096a0 tx=0x7f701c006b40).stop 2023-12-06T03:30:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 msgr2=0x7f7020109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f7020109450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f7010014000 tx=0x7f701000b040).stop 2023-12-06T03:30:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 shutdown_connections 2023-12-06T03:30:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f700c02ff60 0x7f700c032420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 --2- 172.21.15.175:0/1356317977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70200f5410 0x7f7020109450 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 >> 172.21.15.175:0/1356317977 conn(0x7f70200f0b70 msgr2=0x7f70200f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 shutdown_connections 2023-12-06T03:30:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:09.516+0000 7f702851c700 1 -- 172.21.15.175:0/1356317977 wait complete. 2023-12-06T03:30:09.518 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:08.489288+0000 mgr.smithi055.qjkkyb (mgr.14164) 271 : cluster [DBG] pgmap v257: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:09.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:09 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:09.515277+0000 mon.smithi055 (mon.0) 321 : audit [DBG] from='client.? 172.21.15.175:0/1356317977' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:10.902 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:10.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:11.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:11 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:10.489567+0000 mgr.smithi055.qjkkyb (mgr.14164) 272 : cluster [DBG] pgmap v258: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:12.079 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3155438725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 msgr2=0x7f8c580f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:12.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 --2- 172.21.15.175:0/3155438725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c580f6930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f8c48009fb0 tx=0x7f8c48009b30).stop 2023-12-06T03:30:12.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3155438725 shutdown_connections 2023-12-06T03:30:12.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 --2- 172.21.15.175:0/3155438725 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c580f6930 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:12.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3155438725 >> 172.21.15.175:0/3155438725 conn(0x7f8c580f1c70 msgr2=0x7f8c580f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:12.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3155438725 shutdown_connections 2023-12-06T03:30:12.080 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.078+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3155438725 wait complete. 2023-12-06T03:30:12.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c5e5c7700 1 Processor -- start 2023-12-06T03:30:12.081 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c5e5c7700 1 -- start start 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c5e5c7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c58106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c5e5c7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c58106c20 con 0x7f8c580f6510 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c58106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c58106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58082/0 (socket says 172.21.15.175:58082) 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.079+0000 7f8c57fff700 1 -- 172.21.15.175:0/3109317010 learned_addr learned my addr 172.21.15.175:0/3109317010 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c57fff700 1 -- 172.21.15.175:0/3109317010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48005040 con 0x7f8c580f6510 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c57fff700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c58106730 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8c48013510 tx=0x7f8c4800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c48009320 con 0x7f8c580f6510 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c58106e20 con 0x7f8c580f6510 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c48013f00 con 0x7f8c580f6510 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c48014d90 con 0x7f8c580f6510 2023-12-06T03:30:12.082 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.080+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c581072b0 con 0x7f8c580f6510 2023-12-06T03:30:12.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.081+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8c480095f0 con 0x7f8c580f6510 2023-12-06T03:30:12.083 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.081+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c58045ad0 con 0x7f8c580f6510 2023-12-06T03:30:12.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.081+0000 7f8c54ff9700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c4002ff60 0x7f8c40032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:12.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.081+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8c48040fd0 con 0x7f8c580f6510 2023-12-06T03:30:12.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.081+0000 7f8c577fe700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c4002ff60 0x7f8c40032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:12.084 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.082+0000 7f8c577fe700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c4002ff60 0x7f8c40032420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f8c4c0096a0 tx=0x7f8c4c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:12.087 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.086+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8c48014420 con 0x7f8c580f6510 2023-12-06T03:30:12.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.292+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8c580fbf60 con 0x7f8c580f6510 2023-12-06T03:30:12.293 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.292+0000 7f8c54ff9700 1 -- 172.21.15.175:0/3109317010 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8c4801d7f0 con 0x7f8c580f6510 2023-12-06T03:30:12.294 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:12.294 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:12.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c4002ff60 msgr2=0x7f8c40032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:12.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c4002ff60 0x7f8c40032420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f8c4c0096a0 tx=0x7f8c4c006b40).stop 2023-12-06T03:30:12.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 msgr2=0x7f8c58106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:12.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c58106730 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f8c48013510 tx=0x7f8c4800b040).stop 2023-12-06T03:30:12.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 shutdown_connections 2023-12-06T03:30:12.295 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8c4002ff60 0x7f8c40032420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:12.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 --2- 172.21.15.175:0/3109317010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c580f6510 0x7f8c58106730 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:12.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 >> 172.21.15.175:0/3109317010 conn(0x7f8c580f1c70 msgr2=0x7f8c580f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:12.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 shutdown_connections 2023-12-06T03:30:12.296 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:12.294+0000 7f8c5e5c7700 1 -- 172.21.15.175:0/3109317010 wait complete. 2023-12-06T03:30:12.296 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:12.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:12 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:12.293569+0000 mon.smithi055 (mon.0) 322 : audit [DBG] from='client.? 172.21.15.175:0/3109317010' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:13.674 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:13.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:13.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:13 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:12.489854+0000 mgr.smithi055.qjkkyb (mgr.14164) 273 : cluster [DBG] pgmap v259: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:14.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 -- 172.21.15.175:0/1338341864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 msgr2=0x7f8da80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:14.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 --2- 172.21.15.175:0/1338341864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80f5830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f8d9c009fb0 tx=0x7f8d9c009b30).stop 2023-12-06T03:30:14.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 -- 172.21.15.175:0/1338341864 shutdown_connections 2023-12-06T03:30:14.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 --2- 172.21.15.175:0/1338341864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80f5830 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:14.856 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 -- 172.21.15.175:0/1338341864 >> 172.21.15.175:0/1338341864 conn(0x7f8da80f0b70 msgr2=0x7f8da80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:14.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 -- 172.21.15.175:0/1338341864 shutdown_connections 2023-12-06T03:30:14.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 -- 172.21.15.175:0/1338341864 wait complete. 2023-12-06T03:30:14.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.854+0000 7f8daddaa700 1 Processor -- start 2023-12-06T03:30:14.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8daddaa700 1 -- start start 2023-12-06T03:30:14.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8daddaa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:14.857 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8daddaa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da80fb820 con 0x7f8da80f5410 2023-12-06T03:30:14.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:14.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:58102/0 (socket says 172.21.15.175:58102) 2023-12-06T03:30:14.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8da77fe700 1 -- 172.21.15.175:0/1763490830 learned_addr learned my addr 172.21.15.175:0/1763490830 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:14.859 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8da77fe700 1 -- 172.21.15.175:0/1763490830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.855+0000 7f8da77fe700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80fd3a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8d9c013510 tx=0x7f8d9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d9c009320 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da80fb9c0 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d9c013f00 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d9c014d90 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da80fbe50 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8d9c0095f0 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.856+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8059100 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.857+0000 7f8d8ffff700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8d9002ff60 0x7f8d90032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.857+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8d9c041080 con 0x7f8da80f5410 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.857+0000 7f8da6ffd700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8d9002ff60 0x7f8d90032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:14.860 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.857+0000 7f8da6ffd700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8d9002ff60 0x7f8d90032420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f8d980096a0 tx=0x7f8d98006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:14.863 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:14.862+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8d9c014420 con 0x7f8da80f5410 2023-12-06T03:30:15.068 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.067+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8da80fae60 con 0x7f8da80f5410 2023-12-06T03:30:15.070 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.069+0000 7f8d8ffff700 1 -- 172.21.15.175:0/1763490830 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f8d9c01d7f0 con 0x7f8da80f5410 2023-12-06T03:30:15.071 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:15.071 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8d9002ff60 msgr2=0x7f8d90032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8d9002ff60 0x7f8d90032420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f8d980096a0 tx=0x7f8d98006b40).stop 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 msgr2=0x7f8da80fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80fd3a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f8d9c013510 tx=0x7f8d9c00b040).stop 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 shutdown_connections 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f8d9002ff60 0x7f8d90032420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:15.072 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 --2- 172.21.15.175:0/1763490830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8da80f5410 0x7f8da80fd3a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:15.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 >> 172.21.15.175:0/1763490830 conn(0x7f8da80f0b70 msgr2=0x7f8da80f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:15.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 shutdown_connections 2023-12-06T03:30:15.073 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:15.071+0000 7f8daddaa700 1 -- 172.21.15.175:0/1763490830 wait complete. 2023-12-06T03:30:15.073 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:15.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:15 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:14.490184+0000 mgr.smithi055.qjkkyb (mgr.14164) 274 : cluster [DBG] pgmap v260: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:16.444 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:16.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:16.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:16 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:15.070510+0000 mon.smithi055 (mon.0) 323 : audit [DBG] from='client.? 172.21.15.175:0/1763490830' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:17.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:17 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:16.490455+0000 mgr.smithi055.qjkkyb (mgr.14164) 275 : cluster [DBG] pgmap v261: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 -- 172.21.15.175:0/1729126332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 msgr2=0x7f7a340f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 --2- 172.21.15.175:0/1729126332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a340f90f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f7a28009fb0 tx=0x7f7a28009b30).stop 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 -- 172.21.15.175:0/1729126332 shutdown_connections 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 --2- 172.21.15.175:0/1729126332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a340f90f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 -- 172.21.15.175:0/1729126332 >> 172.21.15.175:0/1729126332 conn(0x7f7a340f0b70 msgr2=0x7f7a340f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 -- 172.21.15.175:0/1729126332 shutdown_connections 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 -- 172.21.15.175:0/1729126332 wait complete. 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 Processor -- start 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.608+0000 7f7a3955d700 1 -- start start 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a3955d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a341066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:17.610 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a3955d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34106bd0 con 0x7f7a34060280 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a32ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a341066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a32ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a341066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40236/0 (socket says 172.21.15.175:40236) 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a32ffd700 1 -- 172.21.15.175:0/1970399441 learned_addr learned my addr 172.21.15.175:0/1970399441 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a32ffd700 1 -- 172.21.15.175:0/1970399441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28005040 con 0x7f7a34060280 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.609+0000 7f7a32ffd700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a341066e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f7a28013510 tx=0x7f7a2800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a28009320 con 0x7f7a34060280 2023-12-06T03:30:17.612 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34106dd0 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a28013f00 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a28014dc0 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34107260 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7a28014420 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.610+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34045ad0 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.611+0000 7f7a1b7fe700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1c02ff60 0x7f7a1c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.611+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f7a280410d0 con 0x7f7a34060280 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.611+0000 7f7a327fc700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1c02ff60 0x7f7a1c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:17.613 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.611+0000 7f7a327fc700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1c02ff60 0x7f7a1c032420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f7a240096a0 tx=0x7f7a24006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:17.617 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.616+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7a280095f0 con 0x7f7a34060280 2023-12-06T03:30:17.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.824+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7a34108110 con 0x7f7a34060280 2023-12-06T03:30:17.825 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.824+0000 7f7a1b7fe700 1 -- 172.21.15.175:0/1970399441 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7a2801cb10 con 0x7f7a34060280 2023-12-06T03:30:17.826 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:17.826 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:17.827 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1c02ff60 msgr2=0x7f7a1c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1c02ff60 0x7f7a1c032420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f7a240096a0 tx=0x7f7a24006b40).stop 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 msgr2=0x7f7a341066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a341066e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f7a28013510 tx=0x7f7a2800b040).stop 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 shutdown_connections 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7f7a1c02ff60 0x7f7a1c032420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 --2- 172.21.15.175:0/1970399441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a34060280 0x7f7a341066e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 >> 172.21.15.175:0/1970399441 conn(0x7f7a340f0b70 msgr2=0x7f7a340f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 shutdown_connections 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:17.826+0000 7f7a3955d700 1 -- 172.21.15.175:0/1970399441 wait complete. 2023-12-06T03:30:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:18.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:18 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:17.825628+0000 mon.smithi055 (mon.0) 324 : audit [DBG] from='client.? 172.21.15.175:0/1970399441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:19.229 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:19.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:19.450 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:19 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:18.490745+0000 mgr.smithi055.qjkkyb (mgr.14164) 276 : cluster [DBG] pgmap v262: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:20.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.423+0000 7fcc48227700 1 -- 172.21.15.175:0/2740615378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc400f5410 msgr2=0x7fcc400f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:20.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.423+0000 7fcc48227700 1 --2- 172.21.15.175:0/2740615378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc400f5410 0x7fcc400f5830 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fcc34009fb0 tx=0x7fcc34009b30).stop 2023-12-06T03:30:20.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.423+0000 7fcc48227700 1 -- 172.21.15.175:0/2740615378 shutdown_connections 2023-12-06T03:30:20.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.423+0000 7fcc48227700 1 --2- 172.21.15.175:0/2740615378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc400f5410 0x7fcc400f5830 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:20.426 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.423+0000 7fcc48227700 1 -- 172.21.15.175:0/2740615378 >> 172.21.15.175:0/2740615378 conn(0x7fcc400f0b70 msgr2=0x7fcc400f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc48227700 1 -- 172.21.15.175:0/2740615378 shutdown_connections 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc48227700 1 -- 172.21.15.175:0/2740615378 wait complete. 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc48227700 1 Processor -- start 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc48227700 1 -- start start 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc48227700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 0x7fcc40109560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc48227700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc40109a50 con 0x7fcc40060200 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc45fc3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 0x7fcc40109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc45fc3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 0x7fcc40109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40256/0 (socket says 172.21.15.175:40256) 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.424+0000 7fcc45fc3700 1 -- 172.21.15.175:0/322765851 learned_addr learned my addr 172.21.15.175:0/322765851 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc45fc3700 1 -- 172.21.15.175:0/322765851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34005040 con 0x7fcc40060200 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc45fc3700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 0x7fcc40109560 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fcc34009f80 tx=0x7fcc34009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc340136c0 con 0x7fcc40060200 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc40109b90 con 0x7fcc40060200 2023-12-06T03:30:20.427 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc340095f0 con 0x7fcc40060200 2023-12-06T03:30:20.428 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc34014420 con 0x7fcc40060200 2023-12-06T03:30:20.428 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.425+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc40109fc0 con 0x7fcc40060200 2023-12-06T03:30:20.428 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.426+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc40045ad0 con 0x7fcc40060200 2023-12-06T03:30:20.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.426+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fcc34025060 con 0x7fcc40060200 2023-12-06T03:30:20.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.426+0000 7fcc327fc700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc2c02ff60 0x7fcc2c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:20.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.426+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fcc34040e60 con 0x7fcc40060200 2023-12-06T03:30:20.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.426+0000 7fcc457c2700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc2c02ff60 0x7fcc2c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:20.429 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.427+0000 7fcc457c2700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc2c02ff60 0x7fcc2c032420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fcc3c0096a0 tx=0x7fcc3c006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:20.432 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.431+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcc3401cb60 con 0x7fcc40060200 2023-12-06T03:30:20.638 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.637+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcc400fae60 con 0x7fcc40060200 2023-12-06T03:30:20.639 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.638+0000 7fcc327fc700 1 -- 172.21.15.175:0/322765851 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fcc3400f280 con 0x7fcc40060200 2023-12-06T03:30:20.639 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:20.639 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:20.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc2c02ff60 msgr2=0x7fcc2c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:20.640 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc2c02ff60 0x7fcc2c032420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fcc3c0096a0 tx=0x7fcc3c006b40).stop 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 msgr2=0x7fcc40109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 0x7fcc40109560 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fcc34009f80 tx=0x7fcc34009b30).stop 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 shutdown_connections 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7fcc2c02ff60 0x7fcc2c032420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 --2- 172.21.15.175:0/322765851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc40060200 0x7fcc40109560 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 >> 172.21.15.175:0/322765851 conn(0x7fcc400f0b70 msgr2=0x7fcc400f16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 shutdown_connections 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:20.639+0000 7fcc48227700 1 -- 172.21.15.175:0/322765851 wait complete. 2023-12-06T03:30:20.641 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:20.491086+0000 mgr.smithi055.qjkkyb (mgr.14164) 277 : cluster [DBG] pgmap v263: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:21.950 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:21 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:20.638898+0000 mon.smithi055 (mon.0) 325 : audit [DBG] from='client.? 172.21.15.175:0/322765851' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:22.050 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2023-12-06T03:30:22.050 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5555246c-93e6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-06T03:30:23.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 -- 172.21.15.175:0/3167134289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 msgr2=0x7fb0080f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:23.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 --2- 172.21.15.175:0/3167134289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb0080f6930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7faffc009fb0 tx=0x7faffc009b30).stop 2023-12-06T03:30:23.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 -- 172.21.15.175:0/3167134289 shutdown_connections 2023-12-06T03:30:23.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 --2- 172.21.15.175:0/3167134289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb0080f6930 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:23.173 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 -- 172.21.15.175:0/3167134289 >> 172.21.15.175:0/3167134289 conn(0x7fb0080f1c70 msgr2=0x7fb0080f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 -- 172.21.15.175:0/3167134289 shutdown_connections 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.170+0000 7fb00cdda700 1 -- 172.21.15.175:0/3167134289 wait complete. 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00cdda700 1 Processor -- start 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00cdda700 1 -- start start 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00cdda700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb008106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00cdda700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008106c20 con 0x7fb0080f6510 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb008106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb008106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.175:40274/0 (socket says 172.21.15.175:40274) 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00659c700 1 -- 172.21.15.175:0/3881616670 learned_addr learned my addr 172.21.15.175:0/3881616670 (peer_addr_for_me v2:172.21.15.175:0/0) 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00659c700 1 -- 172.21.15.175:0/3881616670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc005040 con 0x7fb0080f6510 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.171+0000 7fb00659c700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb008106730 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7faffc013510 tx=0x7faffc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.172+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faffc009320 con 0x7fb0080f6510 2023-12-06T03:30:23.174 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.172+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb008106e20 con 0x7fb0080f6510 2023-12-06T03:30:23.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.172+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(25 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faffc013f00 con 0x7fb0080f6510 2023-12-06T03:30:23.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.172+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faffc014d90 con 0x7fb0080f6510 2023-12-06T03:30:23.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.172+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081072b0 con 0x7fb0080f6510 2023-12-06T03:30:23.175 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.173+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7faffc0263a0 con 0x7fb0080f6510 2023-12-06T03:30:23.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.173+0000 7faff6ffd700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faff002ff60 0x7faff0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-12-06T03:30:23.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.173+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb008045ad0 con 0x7fb0080f6510 2023-12-06T03:30:23.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.173+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7faffc0413e0 con 0x7fb0080f6510 2023-12-06T03:30:23.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.173+0000 7fb005d9b700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faff002ff60 0x7faff0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-12-06T03:30:23.176 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.174+0000 7fb005d9b700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faff002ff60 0x7faff0032420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7faff80096a0 tx=0x7faff8006b40).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-06T03:30:23.179 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.178+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7faffc01ccf0 con 0x7fb0080f6510 2023-12-06T03:30:23.389 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.388+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb0080fc050 con 0x7fb0080f6510 2023-12-06T03:30:23.390 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.388+0000 7faff6ffd700 1 -- 172.21.15.175:0/3881616670 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7faffc02b030 con 0x7fb0080f6510 2023-12-06T03:30:23.390 INFO:teuthology.orchestra.run.smithi175.stdout: 2023-12-06T03:30:23.390 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"5555246c-93e6-11ee-95a2-87774f69a715","modified":"2023-12-06T03:20:23.500496Z","created":"2023-12-06T03:20:23.500496Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi055","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-06T03:30:23.391 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faff002ff60 msgr2=0x7faff0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faff002ff60 0x7faff0032420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7faff80096a0 tx=0x7faff8006b40).stop 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 msgr2=0x7fb008106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb008106730 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7faffc013510 tx=0x7faffc00b040).stop 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 shutdown_connections 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:6800/785191101,v1:172.21.15.55:6801/785191101] conn(0x7faff002ff60 0x7faff0032420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 --2- 172.21.15.175:0/3881616670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0080f6510 0x7fb008106730 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 >> 172.21.15.175:0/3881616670 conn(0x7fb0080f1c70 msgr2=0x7fb0080f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 shutdown_connections 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:2023-12-06T03:30:23.390+0000 7fb00cdda700 1 -- 172.21.15.175:0/3881616670 wait complete. 2023-12-06T03:30:23.392 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2023-12-06T03:30:23.783 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 638, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds 2023-12-06T03:30:23.785 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-06T03:30:23.785 DEBUG:teuthology.orchestra.run.smithi055:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-06T03:30:23.812 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-06T03:30:23.830 INFO:tasks.cephadm:Stopping all daemons... 2023-12-06T03:30:23.830 INFO:tasks.cephadm.mon.smithi055:Stopping mon.smithi055... 2023-12-06T03:30:23.830 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055 2023-12-06T03:30:23.886 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: cluster 2023-12-06T03:30:22.491399+0000 mgr.smithi055.qjkkyb (mgr.14164) 278 : cluster [DBG] pgmap v264: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-06T03:30:23.886 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:22.557901+0000 mon.smithi055 (mon.0) 326 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/mirror_snapshot_schedule"}]: dispatch 2023-12-06T03:30:23.887 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:22.560808+0000 mon.smithi055 (mon.0) 327 : audit [INF] from='mgr.14164 172.21.15.55:0/3621304158' entity='mgr.smithi055.qjkkyb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi055.qjkkyb/trash_purge_schedule"}]: dispatch 2023-12-06T03:30:23.887 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: audit 2023-12-06T03:30:23.389549+0000 mon.smithi055 (mon.0) 328 : audit [DBG] from='client.? 172.21.15.175:0/3881616670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-06T03:30:24.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 systemd[1]: Stopping Ceph mon.smithi055 for 5555246c-93e6-11ee-95a2-87774f69a715... 2023-12-06T03:30:24.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:30:23.965+0000 7effcb9c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi055 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2023-12-06T03:30:24.200 INFO:journalctl@ceph.mon.smithi055.smithi055.stdout:Dec 06 03:30:23 smithi055 ceph-5555246c-93e6-11ee-95a2-87774f69a715-mon.smithi055[111758]: debug 2023-12-06T03:30:23.965+0000 7effcb9c8700 -1 mon.smithi055@0(leader) e1 *** Got Signal Terminated *** 2023-12-06T03:30:24.580 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi055.service' 2023-12-06T03:30:24.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-06T03:30:24.683 INFO:tasks.cephadm.mon.smithi055:Stopped mon.smithi055 2023-12-06T03:30:24.683 INFO:tasks.cephadm.mon.smithi175:Stopping mon.smithi175... 2023-12-06T03:30:24.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175 2023-12-06T03:30:24.718 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175.service: Unit ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175.service not loaded. 2023-12-06T03:30:24.719 DEBUG:teuthology.orchestra.run:got remote process result: 5 2023-12-06T03:30:24.720 ERROR:tasks.cephadm:Failed to stop "ceph.mon.smithi175" Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 512, in ceph_bootstrap yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 638, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 531, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/cephadmunit.py", line 154, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi175 with status 5: 'sudo systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175' 2023-12-06T03:30:24.721 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-06T03:30:24.724 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/remote/ubuntu@smithi055.front.sepia.ceph.com/crash 2023-12-06T03:30:24.725 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/crash -- . 2023-12-06T03:30:24.770 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/remote/ubuntu@smithi175.front.sepia.ceph.com/crash 2023-12-06T03:30:24.771 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/crash -- . 2023-12-06T03:30:24.816 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/5555246c-93e6-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-06T03:30:24.816 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2023-12-06T03:30:24.817 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-06T03:30:24.817 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-06T03:30:24.855 INFO:teuthology.orchestra.run.smithi055.stderr:grep: /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-12-06T03:30:24.858 INFO:tasks.cephadm:Compressing logs... 2023-12-06T03:30:24.858 DEBUG:teuthology.orchestra.run.smithi055:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-06T03:30:24.901 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-06T03:30:24.931 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-06T03:30:24.931 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-06T03:30:24.932 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: 96.1% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-06T03:30:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-06T03:30:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:: No such file or directory 2023-12-06T03:30:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mgr.smithi055.qjkkyb.log 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi175.stderr: 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m0.014s 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m0.008s 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.014s 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-volume.log 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mon.smithi055.log 2023-12-06T03:30:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mgr.smithi055.qjkkyb.log: /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-volume.log: /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mon.smithi055.log: 88.3% -- replaced with /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-volume.log.gz 2023-12-06T03:30:24.941 INFO:teuthology.orchestra.run.smithi055.stderr: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-06T03:30:25.016 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mgr.smithi055.qjkkyb.log: file size changed while zipping 2023-12-06T03:30:25.017 INFO:teuthology.orchestra.run.smithi055.stderr: 92.6% -- replaced with /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mgr.smithi055.qjkkyb.log.gz 2023-12-06T03:30:25.090 INFO:teuthology.orchestra.run.smithi055.stderr: 90.6% -- replaced with /var/log/ceph/5555246c-93e6-11ee-95a2-87774f69a715/ceph-mon.smithi055.log.gz 2023-12-06T03:30:25.092 INFO:teuthology.orchestra.run.smithi055.stderr: 2023-12-06T03:30:25.092 INFO:teuthology.orchestra.run.smithi055.stderr:real 0m0.172s 2023-12-06T03:30:25.092 INFO:teuthology.orchestra.run.smithi055.stderr:user 0m0.253s 2023-12-06T03:30:25.092 INFO:teuthology.orchestra.run.smithi055.stderr:sys 0m0.022s 2023-12-06T03:30:25.093 INFO:tasks.cephadm:Archiving logs... 2023-12-06T03:30:25.093 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/remote/ubuntu@smithi055.front.sepia.ceph.com/log 2023-12-06T03:30:25.094 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-06T03:30:25.223 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/remote/ubuntu@smithi175.front.sepia.ceph.com/log 2023-12-06T03:30:25.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-06T03:30:25.261 INFO:tasks.cephadm:Removing cluster... 2023-12-06T03:30:25.261 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5555246c-93e6-11ee-95a2-87774f69a715 --force 2023-12-06T03:31:20.457 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5555246c-93e6-11ee-95a2-87774f69a715 --force 2023-12-06T03:31:20.776 INFO:tasks.cephadm:Removing cephadm ... 2023-12-06T03:31:20.776 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-06T03:31:20.800 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-06T03:31:20.819 INFO:tasks.cephadm:Teardown complete 2023-12-06T03:31:20.819 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 512, in ceph_bootstrap yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 638, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 1463, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 1376, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 531, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/cephadmunit.py", line 154, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi175 with status 5: 'sudo systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175' 2023-12-06T03:31:20.897 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=104d152be7cc4d548bc382f8af8a93d3 Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 512, in ceph_bootstrap yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 638, in ceph_mons while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (181) after waiting for 180 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 1463, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 1376, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_d03fca4f35d43f61b4679936aef7bd2f01cced0b/qa/tasks/cephadm.py", line 531, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/cephadmunit.py", line 154, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi175 with status 5: 'sudo systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175' 2023-12-06T03:31:20.902 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-06T03:31:20.912 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-06T03:31:20.922 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-06T03:31:20.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-06T03:31:20.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-06T03:31:21.014 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-06T03:31:21.021 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-06T03:31:21.022 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T03:31:21.024 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T03:31:21.041 INFO:teuthology.orchestra.run.smithi175.stderr:bash: ntpq: command not found 2023-12-06T03:31:21.044 INFO:teuthology.orchestra.run.smithi055.stderr:bash: ntpq: command not found 2023-12-06T03:31:21.045 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T03:31:21.045 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2023-12-06T03:31:21.046 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 +62us[ +62us] +/- 71ms 2023-12-06T03:31:21.046 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -607us[ -612us] +/- 49ms 2023-12-06T03:31:21.046 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +797us[ +797us] +/- 75ms 2023-12-06T03:31:21.046 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:31:21.049 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T03:31:21.049 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2023-12-06T03:31:21.049 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 6 +76us[ +78us] +/- 71ms 2023-12-06T03:31:21.049 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -577us[ -575us] +/- 49ms 2023-12-06T03:31:21.049 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +819us[ +822us] +/- 75ms 2023-12-06T03:31:21.049 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T03:31:21.050 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-06T03:31:21.059 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-06T03:31:21.059 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-06T03:31:21.070 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-06T03:31:21.195 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-06T03:31:21.284 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T03:31:21.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:31:21.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T03:31:21.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:31:21.356 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-06T03:31:21.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-06T03:31:21.376 INFO:teuthology.task.internal:Duration was 1416.590613 seconds 2023-12-06T03:31:21.376 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-06T03:31:21.386 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-06T03:31:21.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-06T03:31:21.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-06T03:31:21.442 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T03:31:21.447 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T03:31:21.712 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-06T03:31:21.713 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2023-12-06T03:31:21.713 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-06T03:31:21.742 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2023-12-06T03:31:21.743 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-06T03:31:21.765 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-06T03:31:21.765 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-06T03:31:21.786 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-06T03:31:21.851 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-06T03:31:21.851 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-06T03:31:21.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-06T03:31:21.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-06T03:31:21.960 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-06T03:31:21.961 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-06T03:31:21.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-06T03:31:22.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-06T03:31:22.038 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-06T03:31:22.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-06T03:31:22.071 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2023-12-06T03:31:22.088 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2023-12-06T03:31:22.119 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-06T03:31:22.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:31:22.156 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-06T03:31:22.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T03:31:22.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-06T03:31:22.187 INFO:teuthology.task.internal:Transferring archived files... 2023-12-06T03:31:22.187 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/remote/smithi055 2023-12-06T03:31:22.188 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-06T03:31:22.244 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630/remote/smithi175 2023-12-06T03:31:22.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-06T03:31:22.285 INFO:teuthology.task.internal:Removing archive directory... 2023-12-06T03:31:22.285 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-06T03:31:22.288 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-06T03:31:22.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-06T03:31:22.348 INFO:teuthology.task.internal:Not uploading archives. 2023-12-06T03:31:22.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-06T03:31:22.358 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-06T03:31:22.359 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-06T03:31:22.361 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-06T03:31:22.381 INFO:teuthology.orchestra.run.smithi055.stdout: 526086 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 6 03:31 /home/ubuntu/cephtest 2023-12-06T03:31:22.393 INFO:teuthology.orchestra.run.smithi175.stdout: 525410 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 6 03:31 /home/ubuntu/cephtest 2023-12-06T03:31:22.395 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-06T03:31:22.406 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-06T03:31:22.427 INFO:teuthology.nuke:Checking targets against current locks 2023-12-06T03:31:22.451 DEBUG:teuthology.nuke:shortname: smithi055 2023-12-06T03:31:22.451 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T03:31:22.475 DEBUG:teuthology.nuke:shortname: smithi175 2023-12-06T03:31:22.476 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T03:31:22.480 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 02:59:39.046450', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDcA0LeUUrlefBBmmNRRUNrVs3nFmowXMKxFjHDEtrUlVsmfXtKHhFIpQjAilfPIkOAyAAeZ4HqdrBlG9BKkpB4='} 2023-12-06T03:31:22.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-05_19:00:15-rados-wip-yuri6-testing-2023-12-05-0753-pacific-distro-default-smithi/7478630', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 02:59:39.048003', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTlPtw2z9JXlGBbJPqi4iTdHHOIH9ljBicL5hqG0DFhzKadshu9Kwap24oaWBhDZJ0D4laXPRyqYYmwe/5LXIQ='} 2023-12-06T03:31:22.530 INFO:teuthology.orchestra.console.smithi055:Power off 2023-12-06T03:31:22.530 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T03:31:22.557 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2023-12-06T03:31:22.558 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:22.573 INFO:teuthology.orchestra.console.smithi175:Power off 2023-12-06T03:31:22.574 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T03:31:22.583 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2023-12-06T03:31:22.596 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2023-12-06T03:31:22.597 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:22.623 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-12-06T03:31:26.584 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:26.624 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:26.696 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2023-12-06T03:31:26.734 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-12-06T03:31:30.698 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:30.735 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:30.810 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2023-12-06T03:31:30.846 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2023-12-06T03:31:34.814 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:34.847 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T03:31:34.926 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2023-12-06T03:31:34.958 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2023-12-06T03:31:35.028 INFO:teuthology.orchestra.console.smithi055:Power off completed 2023-12-06T03:31:35.059 INFO:teuthology.orchestra.console.smithi175:Power off completed 2023-12-06T03:31:35.166 INFO:teuthology.run:Summary data: description: rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} duration: 1416.5906126499176 failure_reason: 'Command failed on smithi175 with status 5: ''sudo systemctl stop ceph-5555246c-93e6-11ee-95a2-87774f69a715@mon.smithi175''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=104d152be7cc4d548bc382f8af8a93d3 status: fail success: false 2023-12-06T03:31:35.166 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T03:31:35.251 INFO:teuthology.run:FAIL